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ABSTRACT 

This study was designed to explore the effect of 
selected test and background variables on the pooled within-school 
relationship between Graduate Management Admission Test (GNAT) scores 
and first year grade average (FYA), and te assess the potential role 
of selected Test of English as a Foreign Language-related variables 
as supplemental predictors of FYA for foreign students. Data were 
supplied by 59 United States schools of management for 1,762 foreign 
non-native speakers (English second language) and 157 foreign native 
speakers (English primary language). Continuous variables were 
standardised by school— that is, expressed as deviations from school 
means in school standard deviation units — >>nd then pooled for 
analysis. It was found that the mean relative standing of various 
country-contingents in terms of first year grades tended to 
correspond more closely with relative standing on the less valid 
verbal measure than with standing on the more valid quantitative 
measure. One conclusion was that for these samples of non-native 
speakers of English, differences in English-language background 
affected (artificially depressed) bo'h performance on the GNAT verbal 
measure and first-year performance in the Masters of Business 
Administration programs. Appendices provide numerous figures, 
scatterplots, and an interim report dated March 1984. (PN) 
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Abstract 



the primary aim of this cooperative study was to assess the role of selected 
Ehglish-proficimcy related test and background variables as moderators of the 
relationship between scores on the Graduate Managem e nt Amission test (GHAT), 
and first year average grade (RDM in staples of fiorai^i mk students. The 
study was guided by working hypotheses specifying that QHO/nh correlations 
would be systematically higher for students with higher levels of English 
proficiency than Cor those with lower levels of EhgUsh proficiency. Levels of 
proficiency were defined operationally (a) fay native-Ehglish speaking vs 
non-native English sp eaking status, (b) score levels en the Ttet of English as 
a Foreign Language (TOEFL), and (c) score levels on a Relative Verbal 
Performance Index (HVFI)— a derived test variable reflecting level of GMKF 
verbal score relative to that expe cted for U.S. examinees with given 
quantitative scons. It was also hypothesised that GMM/FSA correlations would 
be hic^er (a) for students from count ries whose U.S. -bound nationals typically 
earn higher average scores on TOEFL than for students from countries with 
typically lower-scoring student contingents, and (b) for staples that were 
completely homogeneous with respect to ccuitry of citizenship than in more 
general sasples. 

Data were supplied by 59 U.S. schools of m a na gemen t for 1,762 foreicpi 
non-native speakers of English (Etagiish second language . or ESL) and 157 
foreign native speakers (English primary language or EEL). Continuous 
variables (e.g., GHAT verbal and quantitative score, TOEFL total score, FXA, 
and so on) were standardised fay school— that is, e x presse d as deviations from 
school means in school standard deviation uiite— and then pooled for analysis. 

GHKP quantitative scores were found to be more valid than GMKF verbal scores 
for essentially all subgroups of foreign students. Ibr the BL foreign sanple, 
GMRl v/fya and Q/FYA correlations were similar to medians reported by the 
Graduate Management AMssion Council (GMAC) validity study service (VSS) for 
85 sanples of U.S. MBA students; this was true as well for ESL students 
scoring 600 or higher on TOEFL and for those in the qpper two-thirds with 
respect to the BVPI index. GMXF/FX& correlations were higher for students 
from European countries, and from Asian countries with an established 
English-speaking tradition (e.g., India, Malaysia/ the Philippines), than for 
students from other Asian countries (e.g., Taiwan, Japan, Indonesia, Korea) 
and the Middle East. TOEEVHA and GHAT V/T» correlations were similar. 

It was found that the mean relative standing of various country-contingents in 
terms of first year grades tended to cor r espond more closely with relative 
standing on the less valid verbal measure than with standing on the more valid 
quantitative measure. One conclusion was that for these samples of non-native 
speakers of English, differences in English-language background affected 
(arti factually depressed) both performance on tire GMKT verbal measure and 
first-year performance in the MBA pro g ra m s . 

Findings suggested that a set of subgroup prediction systems would likely be 
better than any general system for foreign VEA students. 
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The primary aim of this exploratory cooperative study was to assess 
the potential role of selected BjjHsh-proficiency related test and 
background variables as Moderators of the relationship between scores on the 

J™*"*? ""ST 11 "! * M "» lon ** t first year average grade (fya) 

^""^•f of foreign NBA students. Accordingly, it was concerned primarily 
with whether or not OM^ft correlations tand to be systematically higher 
(lower) for foreign students classi f i e d according to variables identified as 
potential acderators. The study was not designed to investigate questions 
rfc 55 i ^^ di f tiv ?« bia f' * «»P>rabillty of regression systems for 
predetermined classifications of foreign students, but rather to determine 
irtether there are subgroups of foreign students for which Oftx/FYA 
correlations are likely to differ systematically. 

It was hypothesised that QWT/TVi correlations would ba moderated by 
Bnglish-proficiency related variables— that is, that correlations would be 
higher for students with higher levels of English proficiency than for those 
with lower level* of proficiency as defined operationally by (a) English 
primary language (Eft) versus English second language (ESL) status, (b) 
score levels on the Test of English as a foreign language (TOEFL), and (c) 
score levels on the Relative verbal Performance Index (IMPI), a derived test 
variable reflecting level of GMKT verbal score relative to that expected for 
U.S. examinees with given quantitative scores. 

It was also hypothesised that 3IW/FYA correlations would t» higher for 
samples that ware relatively hcnogenecus with respect to finish-language 
background variables, nested in countries of citizenship, than for samples 
that were relatively heterogeneous with respect to such variables; sore 
specifically, that correlations would be higher fix studants from countries 
whose as.-bound nationals typically have higher average scores on TOEFL 
than for students from countries with typically lower-scoring student 
contingents (Exhibit A). It was further hypothesized that omr/PYA 
relationships would tend to be stranger in samples that ware completely 
hcorxjeneous with respect to country of citizenship than in the more general 
classifications. 

Data were supplied by 59 schools of management for 1,762 foreign E5L 
students and 157 foreign EPL students, largely those entering in fall 1982 
(Tables 1 and 2). Some 140 different countries ware represented in the 
sam ple, b ut 36 countries accounted for about 90 percent of the total (Table 
3). TOEFL scores were available for 1,203 of tha foreign ESL students. Means 
of the the total EPL and ESL samples ware significantly different on all 
study variables except OWP-Q scores, sex, and year of birth. Both samples 
had very high quantitative means (35+), but the verbal mean of the ESL 
sample was depressed (24+ as compared to 33+ for the EPL sample). The ESL 
sample was highly selected in terms of English proficiency as measured by 
TOEFL. Some 2J percent of the ESL sample had U.S. undergraduate origins. 
There were marked differences among the leading country-contingents with 
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respect to all stud/ variables. 

The analysis focussed primarily on data for 1,762 foreign ESL students 
that were pooled across all schools after witMn-school standardisation; 
prior to pooling, the continuous predictor and criterion variables were 
subjected to a witMn-echool ■ scale transformation using paraaetera for 
foreign ESL students. Scores fx Eft students wire a scaled with reference 
to the ESL distributions, The various study hypotheses were evaluated using 
pooled, wi thin-school correlation aatrices for the relevant classifications 
of s tudent s. To assess the potential role of TCEFL and TOSFLEVL scores (the 
Dean TCEFL score of all U.S.-bound TQEFL-takers by country, ascribed to 
students from the respective countries) as supplemental predictors these 
scores were added to a battery composed of GHAT verbal and quantitative 
scores. 

Principal findings were as follows: 

o OwyrYA correlations were higher for the BL than for the combine d 
ESL sample (see Table 4 and related discussion); coefficients for ESL 
students were comparable to those reported for general savples of MBA 
students by the Q9C Validity Study Service (VSS), but those for the 
heterogeneous ESL sample were lower. 

o Within the foreign ESL sample, in analyses involving 1,203 students 
with TOOL scores, amr/Rn. coefficients were found to be relatively 
high (coarmrable to VSS medians for general samples) for students 
scoring 600 or higher, but comparatively low in two lower-scoring 
groups (Table 5). 

o In regression analyses basr* on data for the OKr/TOOL sanple, TOEFL 
Total (T-T) and TOEFLWL (t-L) scores had significant weights when 
included in a battery with OKP verbal and quantitative scores. Mien 
T-T was substituted for QOT-V as tile primary verbal predictor, the 
resulting eultiple correlations with m were *^r^j M t to those 
involving V as the principal verbal predictor (Table 6). 

o When students were classified according to level on the Relative 
verbal Performance Index (RVPI), GMKF/T3A correlations were 
relatively Ugh in the two higher RVPI classifications, representing 
1,152 of 1,762 ESL students, than in the lower-scoring classification 
(Table 7 and Table 8), consistent with expectation. 

o GMftT/m correlations were found to be moderated when students were 
classified according to TOEPLEVL, as higher (T-L 550+) or lower (T-L 
< 500) . As hypothesized, correlations were higher for the higher T-L 
than for the lower T-L classifications (Table 9). 

o When students were classified into 23 analysis groups, most of which 
were homogeneous with respjet to country of citizenship, it was found 
that in the majority of groups, OWT-Q/FXA correlations were higher 

ii 



than the oorraapandlng coefficient in the total ESL saaple while the 
PBpoflte was true for MH/IA coefficients (labia 10); (Mflv-v/FYA 
^! ff icianU ^? classified by country tended to be lower than 

to corresponding coefficient in the cabined ESL staple, flue latter 
finding, which was not expected, mm explained statistically by a 
relatively close association between the T-scaled verbal and rn 
roans of the analysis groups-groups with higher T-ecaled FYA asam 
tep g dtob »ft°P? ldt>> ni9h»r moans on tfcs verbal test rather than 
« <*? ^quantitative test (the awe valid predictor of FYA). This 
result Is understandable if it is assueed (a)that differences aaong 
analysis groups in characteristic levels of functicml Engliah 
proficiency, associatad with countries of origin, affected botb 
performance on the on verbal itaas and parforaanoa during the 
first-year in 1BA prograss. 

o &KT/FA coefficients for certain of the analysis groups were 
considerably lower than typical. Both selection-related and 
English-proficiency related factors appear to be involved (see figure 
1, ttble 11, and related discussion). 

o For foreign SSL students with U.S. undergraduate origin, the 
oTSVWfc correlation was found to he coaparabla to the Median 
ooeffi< ?? lt . "Ported for general 1BA sasples by the Gmc 
VSS, but. was quits low in the subgroup with diverse international 
unde r graduate origins. 

The findings suggest that for foreign nationals from rajor English 
speaking societies, whose linguistic, cultural, and educational backgrounds 
are very similar to those of U.S. students, OAT scores are likely to be as 
valid as they are for U.S. students, the targeted tost population. 

For general sanples of foreign ESL students, perforsance on the (MAT 
quantitative Measure does not appear bo be affected by English-language 
background variables; this aaasure appears to saintain its construct 
validity across linguistic-cultural boundaries. «*»«ucc 

However, in sanples of foreign ESL students, the QKT verbal section 
appears to be Measuring differences in the functional EngUsh-lariguage 
ability (English proficiency), associated with countries of ciUsensMpTas 
audi as (in addition to) English-language verbal reasoning ability (the 
test-construct in sasples of U.S. students). And, the relative standing of 
various counti/-contingents in terns of first-year VBk performance (mean 
T-scaled FYA) tended to correspond with their relative standing on the OWT 
verbal measure. 

To the extent that average differences in FYA for students classified 
by country reflect differences in average English language proficiency, 
questions are raised regarding the meaning of the average FYA differences. 
Students with limited English language backgrounds, for example, may know 
more than they are able to demonstrate through classroom participation, 
written examinations, and other assignments. Exploratory use of personal 
assessment techniques would be useful in assessing this possibility. 

iii 
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Generally speaking, study finding* suggest that there *re subgroups 
within the general ESL population for which OAT/m relationships are 
likely to differ systematical!! . A major implication it that a aoderated 
prediction system for subgroups of foreign students is likely to be 
effective than any general — ' - 



A classification (subgrouping) scheae based on country of citizenship 
appears to have considerable promise; most of the moderating effect 
associated with classification by country say be realised by using clusters 
of countries, rather than individual countries, as the basis for operational 
subgroup-classification. An illustrative classification, based on study 
findings, is provided. 

Classification of students according to the English-proficiency related 
test measures also appears to have prcsdse in a soderated system. 



further research is needed (a) to assess £» comparability of 
regression systems for subgroups of foreign students based on the variables 
identified as soderators In this study and (b) to determine the practical 
utility of a soderated prediction system. Given the expected small size of 
foreign ESL samples in individual schools, and the apparent need for a 
roderated-prediction system, a model that is capable of treating data for a 
large number of small samples appears to be necessary to the development of 
such a prediction system. A statistical model based on empirical Bayesian 
concepts, has been applied by Braun and Jones (1981, 1982) in studies 
involving small samples of minority students in several schools of 
management and a number of small graduate depa r tme n tal samples, 
respectively. Shia model would seem to be adaptable for application to the 
complex research problem of developing and testing the utility of a 
moderated-predicticn system for foreign ESL applicants. 

Results of the present study, like those of studies of the 
characteristics and the test performance of foreign nationals taking the 
Graduate Record Examinations (GRE) General Test (Wilson, 1984a, 1984b, 
1982c), and of previous studios of the impact of language background on GMftT 
performance (Powers 1980; Wilson 1982c), indicate that English language 
"verbal ability" tests are not measuring the same construct in samples of 
non-native English speakers as in samples of native speakers, U.S. or other. 
Thus, the verbal scores of U.S. and randomly selected foreign ESL examinees 
ca ??2 t J b L£ sa " 8d to te «»P "ra ble— that is, cannot be assuned to reflect 
valid differences in verbal reasoning ability, fliis is a factor to be 
carefully weighed in the design of future validation research, especially in 
considering study designs that might call for pooling verbal test, data for 
U.S. and foreign 2SL students; differences in construct validity raise 
important questions regarding the interpretation of pooled wi thin-school 
QKiynh correlations based on data for combined U.S. and foreign-ESL 
students. 



iv 
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FACTORS AFFECTING QKF KBSXCIXVE VALIDnY ICR FOKEHH MBA STUDENTS: 

*A EXELORATDRY SItBY 



Kenneth M. Wilson 
Educational Testing Service 
Princeton, KJ 08541 

Introduction 

The Graduate Management Attain Mt (GHAT) is intended for use in 
evaluating the acadesdc qualifications of applicants for admission to graduate 
schools of management. GNNT provides Measures of verbal and quantitative 
reasoning abilities «3«r-V and GMKP-Q) and also reports a total soon* Th& 
examinee population taking GNAT is Bade up predominantly of U.S. citizen*, 'jo 
whom the test is oriented linguistically, culturally, and educationally. 
However, the &AT program also serves foreign nationals—during 1980-81, for 
example, it is estimated (GHAC 1982) that some 27 percent of all examinees 
tested ve*e foreign nationals from more than 125 countries. 

Foreigi examinees differ from U.S. examinees, and among themselves, with 
respect to cultural, educational, end linguistic background variables, nested 
primarily in country of citizenship. Fbr exaaple, managaroent=schwl-bcwnd[ 
foreign nationals from different non-native English speaking countries differ 
markedly in average levels of developed proficiency in English as a s econd 
language as measured by the Vest of BagpLimb as a Fbceign language or TOEFL 
(Wilson 1982a, 1982b, Bowers 1980), which is desigisd for use by foreign 
nationals to demonstrate their English proficiency (EES 1981). 

The average quantitative performance of foreign GHAT examinees for whom 
English is a second language (foreicn-ESL examinees) is conparable to that of 
the general gRat population, but the average verbal performance oJ the group 
(at about the 15th percentile relative to all QKT examinees) is ouch lower 
(GMAC 1982, Powers 1980, Wilson 1982c). The depressed performance of foreigrv- 
esl examinees on GNAT Verbal may be attributed primarily to factors associated 
with their less than native levels of proficiency in English, including 
lower-than-native levels of need of verbal processing; this is evidenced, for 
example, in the lower completion rates of foreign examinees on the GHAT 
(Simott 1980). Similar patterns of d e presse d verbal test performance relative 
to quantitative pe rfo rma n ce hove been fond to be characteristic of foreign- 
ESL examinees who take the Graduate Sscocd Examinations (ORE) General Test 
(see Wilson 1984a, 1984b, 1982c for detailed data). 

There is a substantial body of evidence regarding the predictive validity 
of GHAT scores and other admissions measures, such as the undergraduate GPA, 
in general samples of first-year MA students— for exanple, 85 studies were 
conducted by the Graduate Management Adtadssicn Council (GMAC) validity study 
service (VSS) during the period 1978-79 through* 1980-81 (Hedit and Powers 
1982). However, evidence regarding the validity of GMAT scores for foreign 
nationals, especially non-native English speakers, vrtx> apply for admission to 
U.S. schools of management is limited. 

During the period covered by the 85 general-sample VSS studies , for 
exanple, only six schools submitted data to the GMAC VSS for subgroups of 
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norHJ.S. citizens. In these six studies, the verbal soon means for foreign 
subgroups were f xnsistently loner than those of Their U.S. classmates. GHAT 
scores wire oositivrly correlated with fir st year average grade (ra p in the 
forei^p saegues, end the first year average grade for the foreigp students was 
roughly oosparable to that of domsetic mtodapfca deepite tte disparity in their 
verbal aooce averages. However, the foreign student subgroups typically were 
quite small— too saall to permit reliable estimates of GHW/FXA relatiowhips. 
in addition, tte studies were not designed to control for national origin, 
English language background, undergraduate origin (U.S. vs other), or other 
background variables that say reasonably be e xp e cted to Moderate the 
relationship between GMKF scores and first-year perfocnenoe in MBA program* 
for foreign students.* 

It is reasonable to hypothesise, for example, that in samples of foreign 
esl students the predictive validity of GHKT soores (especially scores on the 
verbal test), may be aoderated by level of Ihgliah proficiency t hat is, in 
sanples of foreiga-ESL students m have acquired a relatively high level of 
Qnglish proficiency, the validity of GKKT scores should tend to be greater 
than for students with relatively low levels of developed Bnglish proficiency 
For these latter students, di ff e r ences in GMKF verbal soores, for exaqple, may 
reflect differences in level of acquired proficiency in English rather than 
differences in level of developed verbal ability. 

Similarly, it is reasonable to hypothesise higher QKT/FA correlations 
for students from non-Bnglish speaking societies that are siidlar in ling- 
uistic-cultural-educational heritage to the United States (e.g., Western 
Europe), end countries in which English is an official antftor academically 
prominent language (e.g., India, Nigeria), than for student* f^cm societies 
whose heritages are less similar (e.g., Asian end Itydeastem countries). And, 
apart from the foregoing, QM|/Tft correlations might be txpected to be higher 
in samples that are henegs nsou s with respect to national origin than in 



♦Research concerned with "moderator" variables has been characterized by lack 
of consensus regarding definition and methodology (see, for example, Journal 
ot Applied Psychology, Vbl. 56, 1972, pp. 245-251, feature Section: Moderator 
variables). However, one consistent theme has involved the notion that 
predictor-criterion correlations are likely to be systematically higher 
(lower) in some subgroups then in others. For exasple, tftere is some evidence 
that test validities tend to be higher for women than for men in a ruber of 
undergraduate and secondary school settings, and sax is said to moderate the 
relationship between academic predictors and criteria (e.g. , Bock, Bar one, & 
Lim 1967). The differences by sex in predictor-criterion relationships are 
presumed to be due to eex differences in attitudes toward academic work, 
persistence, work habits and the like. Similarly, "degree of motivation 19 would 
be expected to moderate the relationship between measures of "aptitude*" and 
measures of "performance"— for staple, aptitude-cerfonnance relationships 
should be stronger Li highly motivated than in poorly motivated groups. This 
study is concerned with the extent to which GM/FXA relationships are system- 
atically moderated by selected continuous and dichotcmous variables that re- 
flect differences in level of English proficiency. 
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saqples that ace heterogeneous in this regard, given the extreme diversity 
among countries with respect to the patterns of English language acquisition 
and use, culture, and educational programs. 

It is also reasonable to expect the predictive validity of the previous 
academic record, as indexed fay the undergraduate grade point average, to be 
higher for foreign nationals vho cospleted their undergraduate work in the 
U.S. than for students with diverse, international undergraduate origins. 



Study Objectives and Design 

The primary aim of this exploratory study w as to assess the role of 
selected test and background variables (such as TOEFL scores, country of 
origin, and undergraduate origin) as moderators of the relationship between 
gnat scores and first year pe rfor mance (Wfh) in sanples of foreign NBA 
students. Accordingly the principal interest it in wheth er or not GMKT/FJGfc 
correlations tend to be hitter (lower) for students classified according to 
these potential moderator variables. Hie study explored the potential utility 
of selected variables both as supplementary predictors and as moderator 
variables. The study wee not designed to investigate predictive bias or 
comparability of regression systems for various subgroups, but rather to 
determine the effect of English proficiency related variables on QAT/KA 
relationships- *hat is whether or not systematic differences in level of 
gmat/fya relationships are likely to obtain for subgroups differing in 
lingjdstic-cultural background. 

Data were obtained, through the cooperation of 59 U.S. schools of manage- 
ment, for foreign NBA students (without regard to their U.S. visa or residency 
status) uho entered in fall 1982, as full-time students, and *ho earned a 
first year grade point average (RA). Participating schools provided GHAT 
scores (verbal, quantitative, and total) and a first year average (FXA) for 
each study-eligible foreign student, plus information on year of birth, sex, 
undergraduate origin (U.S. vs othe r), country of citizenship, and native 
language, and uhen available, TCEFL Total scores. Several schools supplied 
undergraduate GPA (UGEA) . 

As anticipated, the school-level saqples of foreign ESL students were 
all quite small by usual validity study standards. (The median N for the 59 
samples v*as 26, with a range of Ns between 6 and 77; 22 sanples included 30 or 
more students, 24 samples included between 20 and 29 students, and 13 
included fewer than 20). For perspective, only three of 85 general first-year 
samples studied by the GWC VSS during the academic years 1977-78 through 
1979-30 included fewer than 77 students and the mean sample size was 175 
(Hecht & Powers, 1982). 

Collectively, however, the participating schools supplied data for 1,924 
foreign students. Five students could not be classified by country of 
citizenship; of the remaining students, 157 (or 8.2 percent) were foreign EEL 
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studsnts (those vhose reported native or primary language was English), and 
1,762 were foreigrv-ESL students ( non-native Bnglish-speakers for wham English 
was a secondary language), the prinary study sample. 



General Methodological Rationale 

ihe nunber of foreign students in individual MBA program was too small 
to yield reliable estimates of the relationship of GMKT scores to student 
performance or to per^t exploration of the role of background factors either 
as predictors of ormence or moderators of GMKT predictive validity. 
However, given cc%mn6le data sets for a relatively large nusJer of smell 
sanpies of individuals engaged in similar activities but in different settings 
(e.g., different Wk programs) it is possible to drew meaningful in&rences 
regarding the characteristic patterns of relationships among the variables 
(that is the pooled within-schooi ir* rcorrelations of the variables) by 
basing analyse on ccabined data from ai* die setting*. 

Given objectives like thoee of this study, a useful approach to pooling 
data for analysis is to standardise the study variables within each setting 
(school, program, etc.) before pooli n g th a t is, within each school, for 
example, express scores on all variables as deviations from school means in 
school standard deviation units (see, for exasple, Wilson 1979, 1982d, 1964c). 
in these studies, pooled idtMtt-gpcoqp correlations were analysed for relative- 
ly large tutors of saall departmental seniles of ?**fcste or undergraduate 
students with the aim of assessing the characteristic patterns of relation- 
ships b etw ee n Graduate Becocd ftmmhl Iim (GBB) scores (at the General auVbr 
Subject nets) and graduate or undergracbabe grade point average criteria. . 

o Wilson (1979), for example, employed data for 139 graduate' depa rt mental 
samples, from 39 graduate schools, representing more than 20 different 
fields of study to estimate typical patterns of criterion-related validity 
coefficients and regression weights by field* In an analysis involving 54 
departmental saoples from five fields of stud/, it was found that in most 
instances, regression coefficients for GRE predictors based on data for 
individual departments did not deviate significantly from the correspond- 
ing, pooled wi thin-department coefficients* Fooled, wi thin-department 
data were also employed in assessing the criterion-related validity of the 
restructured GBB General Test (Wilson 1982d) in a saople that included 
data for first-year graduate students in 100 d e p a rtm e nt s distributed among 
eight different graduate fields— 59 of the departments were represented by 
between 5 and 9 students* 

o Tfri relationship of item-type part scores on the GRE General Test to 
urdergraduate grades was assessed using pooled, dep art am n tally standard- 
ized data for college senior-level students and recent graduates from 437 
undergraduate depa rt me nt s representing 12 field* of study and the major 
undergraduate suppliers of GRE test takers (Wilson, 1984c). Ihe graduate- 
1 ?vel studies involved exploratory assessment of characteristic predictor- 
criterion relationships for subgroips (for exanple, students classified by 
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sex or ethnic status), and of the relative within-departnent average 
standing of subgroups. 

Pooled within=school correlations wy be thought of as approximating 
"population 19 valueo aromd which the coefficients for saqples from the 
respective schools will vary, due to selection- or saspling-related consider- 
ations (such as restriction of range on the predictor anchor criterion 
variables) as well as context-specific ( situation-specific ) validity-related 
considerations (for exanple, quantitative nthods any be sore heavily 
emphasized in sane school curricula than in others). 

There is reason to believe that such of the variation In observed validi- 
ty coefficients for cannon predictors and criteria across similar settings is 
explained by statistical artifacts rattier than by situation-specific validity- 
related factors. For example, in an analysis of 726 Im school validity 
studies (Lim, B&rnisch, a Dunbar 1981), soae 70 percent* of the variation in 
validity coefficients across studies was attributable to differences in saople 
standard deviations, estimated criterion reliability, and sanple size, 
respectively. Similar findings have been reported for validity studies 
involving cannon selection tests in employment settings (for exaeople, Ferlman, 
Schmidt, & Hunter, 1980). 

The present exploratory study was designed to assess the characteristic 
patterns of wi thin-school relationships aacng standard predictors (that is, 
OAT scores) end a standard criterion variable (naraely, first year average in 
the MBA program, or F») for foreign-ESL students, generally, and in subgroups 
classified according to background variables that on a priori grounds adght be 
expected to M od erate (affect systematically) GHKC/Hk relationships. Results 
of analyses based on pooled, within-sctecl data soy be thought of (a) as 
having generalisable implications for the use and interpretation of GHAT 
scores for foreign students, (b) as providing insight regarding background 
variables that need to be incorporated in the design of operational prediction 
systems for foreign students, and (a) as a useful first step toward the 
development of prediction systems that take into account the specific 
circumstances of individual programs. 



Detailed Description of Study V&riables 

Schools supplied GMKT verbal, quantitative, and total scaled scores and a 
first year average grade (FA) for each student, plus information regarding 
sex (coded male - 1, female - 2), year of birth (inversely related to age), 
undergraduate origin (U.S.- 1, other - 0), country of citizenship, and native 
language (coded English priaary or native lang uage, or EPL - 1, vs English is 
the second language, or ESL - 0). A TOEFL total score was supplied, if 
available, for ea ch st udent. Presence vs absence of TOEFL was treated as a 
nominal variable (TOOL present - 1, not present - 0), labelled YEST0EFL. Only 
21 schools opted to provide data on the undergraduate GBh (U3EA). 

A standard conposite of GNAT verbal and quantitative scores (Q + .6V! 
was included as a special study variable. The weights involved in this 
conposite reflect the ratio of optimal average weights for these two scores as 
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detendned in previously conducted one Viddity Stud/ Service (VSS) analyses 
based on general saqples of students from 25 of the schools participating in 
the present study. This variable is labelled VS9QQNP, for VSS Oa^osite. 



Ehglish--proficiancy Related Variables 

TOEFL total scores (TOBPLTOT) were available for 68 percent of the 
foreign ESL students. The total score on this widely used test of English 
proficiency tends to be correlated moderately highly with GMKMferbal in 
general samples of (SKm/KKTL takers— correlations of approximately .7 have 
been reported for lane swpTee from the general GMKtyTOEFL population 
(Powers, I960; Wilson 1982c). On the strength of this level of relationship 
between GNKT verbal and TOEFL scores, — HPT or MP might be expected to 
have correlations wittL academic criteria similar to those for QKT verbal 
scores. TOEFL total may be thought of both as a potential moderator variable 
and as a myplemsntal predictor of R\* 

Ttao additional Bngliah-praficiency related variables were included in the 
study. One wes intended to reflect characteristic differences among countries 
in the level of devel oped En glish proficiency of their O.S.-goriuate-school- 
bound nationals (SBl LB/EL, KXFUtfL, T-L); the second variable, called the 
Relative Wrfaal Performance Xodeot oc BUFE was developed (Wilson, 1984a) as an 
index of an "English proficiency deficit 91 in the observed GRB verbal perform- 
ance of contingents of fbreigp ESL examinees from different countries. 

TOBFUWL . There are marked differences among countries with respect to 
the TOEFL total means of their U.S.--gradmteHKhc^^XMfid nationals and these 
differences appear to be relatively stable . over time; a correlation of .94 
wes found between national means of exa m in e es in two testi ng ye ars, based on 
data for 129 countries (Wilscn 1982a). Ihe differences in TOOL memos may be 
thou^it of ae reflecting differences among countries of origin in pet te r ns of 
English lang u age a g ylm l tion an d usage end associated differences in the 
general "richness* of the Bngllsh language backgrounds cf students planning to 
study in the United States. Ibr exanplt, exam i n e es from non-native English 
speaking societies in which audi instruction in higher education is in English 
(sudi as India, the Philippines, or Nigeria), or whose native languages rid 
English have numerous cannon elenents (as is the case for many European 
examinees, for exaqple) typically earn such higher TOEFL scores than those 
from, say, Asian or Mid&stern countries uhere relatively little formal 
instruction is conducted in English, and vhere there is substantial linguistic 
distance between native languages and English. 

Exhibit A shows TDEFUEVL values used in the study for students from a 
representative array of countries; TOEFLEVL was available for all students 
(exce pt five for whan country of citizenship was missing). Like TOEFL Total, 
TOEFLEVL nay be useful as a predictor of FX* anchor as a moderator variable. 
For the present study, the mean of the most recent scores of U.S. graduate- 
school-ban! TOEFL e xamin ee s from a given axntry was ascribed to each student 
from that country— thus, the TOEFLEVL score for all students from Thailand was 
472, Algerian nationals were assigned a score of 505, and so on. 
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Exhibit A 



TOEFL Means For Various Contingents of U.S. -Graduate-School-Bound 
Foreign Nationals, by Planned Analysis Group 



Analysis 
group 

01 Algeria 
Kuwait 
Qatar 

Saudi Arabia 

Tunisia 

Yemen 

Iraq 

Libya 

Syria 

Sudan 

Egypt 

Lebanon 

Iran 

Jordan 

02 Thailand 

03 Taiwan 

04 Korea 

05 Japan 

06 Hong Kong 

07 02 - 06 

08 Mexico 



09 



Brazil 

Chile 

Peru 

Argentina 
Costa Rica 
Nicaragua 
Ecuador 
Panama 
Guatamala 
El Salvador 
Uruguay 
Venezuela 
Dominican 
Paraguay 
Colombia 



(10) 08 - 09 



Rep 



TOEFL 
Level* 

505 
448 
422 
443 
497 
466 
454 
448 
491 
474 
478 
501 
456 
466 

472 

514 

513 

504 

505 

521 

515 
524 
510 
552 
524 
497 
502 
504 
532 
512 
550 
493 
«96 
498 
511 
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Analysis TOEFL 

group Level* 

11 Greece 514 
Turkey 510 
Cyprus 499 

12 Pakistan 524 

13 Malaysia 559 

14 India 556 

15 Nigeria 553 

16 Singapore 556 

17 Philippines 594 
(18) 12 - 17 

19 France 570 

20 Luxembourg 600 
Belgium 585 
Norway 576 
Sweden 594 
Germany (FR) 583 
Netherlands 601 
Spain 552 
Italy 549 
Austria 583 
Switzerland 576 
Denmark 594 
Iceland 571 
Finland 582 

(21) 19 - 20 

(22) Other nations 550+ 

(23) Other nations <550 



TOEFL Total means ot U. S. -graduate- 
school-bound nationals tested during 
1977-1979 (Wilson 1982a), ascribed to 
students from the respective countries 
as TOEFLEVL scores. 
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Exhibit A anticipate* the clustering of certain countries for purposes of 
analysis. Note that, generally speaking, grcjping of countries as in Exhibit A 
tends to control for native languages as well as characteristic level of TOEFL 
score. 

ablative Verbal Ferfoaanct Jtoflax . The BMPI is a neasure of the discrep- 
ancy t~tween observed verbal perfodnnce and expected verbal performance "where 
expected pe rfor mance is defined as that «cpected for U.S. examinees with given 
scores quantitative x tests, on nhich the performance of foreign 

examinees appears to be <*avely raffected by linguisticHaaltural back- 
ground factors (for exeegLi, Wilson 1964a, 1962c; Powers I960). In deriving 
this index for the present study, a regression equation for predicting verbal 
free quantitative scores in m appropriate sample of U.S. GHff examinees ves 
used to determine the expected verbal score. 

Hie following equation was employed: 

V-exp - .562 Q 4 13.23.* 

By definition, for the U.S. GMNT exaodnees involved, the mean discrepancy 
between observed and expected OAT verbal is zero, and the standard deviation 
of the distribution of discrepancies is given fay the standard error of 
estimate, vhich for the general saaple was 6.37 points on the ORE verbal 
scale. As used in this study the ROT is a T-ecaled, lin&ar tr an sfo rmation of 
the dist ri bu t i on of expected discrepencies (with seen of aero and standard 
deviation of 6.87) into a distribution with nsan of SO and standard devi- 
ation of 10. Thus, for exaople, RWT * 50 indicates a verbal score exactly 
equal to that pradicted for a U.S. exaninee with a given quantitative score, 
RVPI ■ 40 indicates a verbal score that is lower than predicted fay one 
v standard error (10 points on the transformed scale equal 6.87 points on the 
GWkT scale), RVPI - 55 indicates a verbal score higher than expected by one 
half of a standard error of estimate, and all other RVPI values may be 
similarly interpreted. 

Mean RVPI values for examinees classified by world region and by reported 
language of greatest fluency, and the corresponding GMNT score smeary 
statistics as reported by QRT (1982), are shown in Exhibit B. 



Means, Standard Deviations, 2nd Intercorrelations of the Variables 

Table 1 shows data availability and sunnary statistics for GMAT scores 
and other basic indepen d ent variables for the the total ESL and epl saqples; 
intercorrelations are shown in Table 2. Note that the EPL and ESL samples 



*This equation vas based on means and standard deviations for 156,664 U.S. 
examinees tested during 1980-61 (Q9C 1982) as follows: Verbal mean * 28.29, 
quantitative mean * 26,79, verbal standard deviation - 8.23, and quantitative 
standard deviation * 8.06. EES internal analyses indicate correlations 
between verbal and quantitative scores tend, typically, to be about .55. This 
coefficient was used in deriving the equation. 
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Exhibit B 



GNAT Verbal, Quantitative and Total Score Summary Statistics 
and Relative Verbal Performance Indices by World Region and 

Primary Language 
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Table 1 



and Standard Deviation* for the Foreign ESL and Foreign EPL 
Samples on GMAT and Other Study Variables 



Variable 


ESL 

N 


Students 

Ma an 

nean 


c r\ 

OmU 


N 


EPL Students 
Mean 


S.D 


OflAT-V* 

GMAT-Q 
GMAT -Total* 
VSSCOMP* 


1767 
1767 
1767 
1767 


24.1 
35.4 
495.1 
49. « 


7.8 
8.7 
89.2 
O1.0 


157 
157 
157 
157 


33.5 
35.5 
569.2 
55.6 


8.2 
9.1 
101.7 
12.4 








SEX (M«l;F-0) 
YEAR OF BIRTH 


1766 
1766 


1.18 
55.8 


0.38 
3.6 


157 
157 


1.22 
55.5 


0.42 
4.6 


U.S.-UG*<? 

RVF1* 

TOEFLEVL* 

TOEFLTOT* 

YESTOEFL*# 


1690 
1767 
1762 
1205 
1767 


0.23 
36,9 
529.6 
584.8 

0.68 


0.42 
11.5 
34.2 
43.3 

0.47 


142 
137 
157 
12 
157 


0.35 
50.4 
605.8 
624.2 

0.08 


0.48 
19.7 
34.7 
22.6 

0.27 


*Differences in 


EPL and 


ESL means significant 


at p 


< .003. 





.S. UG m 1, other - 0; ffTOEFL score available - 1, not avail. - 0 



Table 2 



Intertorrelations of Study Variables in Total ESL and EPL Samples 



GMAT GMAT GMAT VSS 
V Q TOT COMP 



SEX BIRTH U.S- RVPI TOEFL TOEFL YES 
YEAR UG LEVEL TOTAL TOEFL 



GMAT-V .295 

GMAT-Q .544 

GMAT-Tot .900 .853 

VSSCOMP . 791 . 944 
SEX(M»l l F-2)-.143 -.244 

BIRTHYR .058 .182 

U.S.-UG@ -.230 -.233 

RVPI .784 -.094 

TOEFLEVL .486 .279 

TOEFLTOT .434 .444 

YLSTOEFL0 -.173 -.046 



.832 
.756 

.977 
.218 
.129 
.267 
.437 
.446 
.502 
,136 



.657 
.909 
.946 

-.235 
.156 

-.259 
.241 
.395 
.497 

-.102 



.033 
.093 
.077 
.088 

.064 
.083 

.010 
.084 
.325 

,039 



.209 
.045 
.167 
.124 
.104 

.062 
-.066 
.048 
.103 
.064 



.093 
.262 
.083 
.168 
.040 
.112 

.102 

.200 
.024 
,065 



.806 
-.323 
.357 
.083 
.025 
.179 
.254 

.370 
.187 
-.172 



.303 
-.094 

.155 

.054 
-.095 

.117 
-.062 

.358 

.003 
-.219 



.648 
.126 
.526 
.382 
.091 
.163 
.103 
.552 
.424 

;/ 



-.051 
.228 
.086 
.158 
-.046 
-.010 
-.521 
-.192 
.023 



Note. Coefficients above the diagonal are for foreign ESL students; those he- 

xS^flnx , a|?0n " e i° T f ° relgn EPL stud e nt *- N * for coefficients involving 
TOEFLTOT do not exceed 1205 for the ESL sample and 12 for the EPL sample. 

0U.S. UG - 1, other - 0; # 1 - TOEFL available, 0 - not available, 
correlation „ t th TOEFL not meaningful. ~ 

1 if '" <Jr J*\ >'V ') " 
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differ significantly on all variables except GMKT quantitative, sex, and year 
of birth. ESL students performed as well on QW quantitative as did EPL 
students but had audi lower ranas on verbal and total, and on the English 
proficiency-related variables. Note that the amn of EPL students on RVPI was 
50.4, indicating verbal performance like that of U.S. examinees with similar 
quantitative scores, while that of ESL students was 36.9, indicating verbal 
performance veil below that expected of U.S. examinees with similarly hic^h 
quantitative scores (13.1 T-scaled points below the expected asan of 50, or 
1.31 standard errors of estimate). 

Both samples were predominantly male in composition; only 18 percent of 
the ESL and 22 percent of the EPL saqple were women. U.S. unde rg raduate 
origins ware reported for about 23 percent of ESL and 35 percent of EPL 
students. 

For perspective in evaluating the mean GMKF scores, the means for all 
U.S. examinees tested during 1960-81 were 26.8 and 28.3 for the verbal and 
quantitative measures, respectively (GMAC 1962). Both of the foreign student 
samples were very highly selected on quantitative ability, relative to the 
GNAT examinee population generally. Moreover, the verbal nan of the foreign 
ESL sanple (24.1) was considerably higher than the mean (approximately 20.0) 
registered by all foreign nationals who took GMKT daring the period 1977 
through 1979 (Wilson 1982c, Powers 1981). Ihus, the foreign ESL as well as 
the foreign EEL students in the study sasple were highly selected or* both 
verbal and quantitative ability, although the foreign ESL sample appears to 
have been somewhat more highly selacted on quantitative ability than on verbal 
ability. Other points of interest include the following: 

o Scores on TOEFL were available for 68 percent of the ESL sample; 12 
EPL students (8 percent) also had TOEFL scores. From the intercorrel- 
ation table it may be seen that for the ESL sanple, the presence or 
absence of a TOEFL score was more closely associated with undergraduate 
origin than with any other variable (poin t biserial coefficient of 
-.521 in the ESL sanple) — absence of TOEFL was associated with U.S. 
origin of the bachelor's degree. 

o The TOEFL Total mean for the 1,20 5 ESL students who did present scores 
was 584.8. For perspective, the TOEFL mean for all GMKT/TOEfli exam- 
inees tested during 1977 - 1979 w as 553 (Wilson 1982b) while the mean 
for all U.S.^radate-sctooi-4xund TOEFL examinees was only 511 (Wilson 
1982a). Ihus, the foreign ESL students in the sanple were relatively 
highly selected in terms of English proficiency. 

o For foreign ESL examinees, the correlation between GMKT verbal and 

quantitative scores is lower than that typically found for U.S. 

examinees (r - .295 as coopered to r - app acimately .55) and that 

reported in Table 2 for foreign EPL examinees (r - .544). 

o By inference from the point biserial coefficients reported in Table 2, 
among ESL students those with U.S. undergraduate origins tend to have 
somewhat higher GHAT verbal scores (r - .093 between U.S.-UG - 1, other 
• 0)) but lower quantitative scores (r - -.262 for the same variable). 
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o ESL students with U.S. indenpaduate origins tended to have higher RVPI 
scores and TOEFL Total scores than others. 

o ESL students without TOEFL scores scored higher on verbal and lower on 
quantitative than those with TOEFL scores; absence of TOEFL scores was 
associated with lower scores on RVPI. 

o In the ESL sample, year of birth (inversely related to age) had low 
positive correlations with all variables except YJSTCEFL— younger 
stude nts sore frequently were not required (by inference) to take 
TOEFL. 

o Negative coefficients between SEX, GHAT scores, TCEFLEVL and T O E F' L 'JOT 
indicate a tendency for women to have slighty lower average scores on 
these variables than nan. However, women had slightly higher RVPI 
means than men. 



Means on Basic Study Variables, by Country 

Some 140 different countries were represented in the study sample by one 
or more students (see Appendix A-l for complete enumeration) . However, 36 
countries that were represented by 10 or more students accounted for slightly 
over 90 percent of the total foreign student sample (ESL plus EFL) . Means on 
the study variables are shown in Table 3 for students from these 36 countries 
which are listed in descending order with respect to mean RVPI. The largest 
contingents came from Taiwan, India, Japan, Korea, Thailand, Mexico, Hong 
Kong, Malaysia, France, Canada, and Nigeria, all of which were represented by 
at least 50 students. The' 157 students who were reported by schools as native 
speakers of English were drawn heavily from the Canadian, British, South 
African and Jamaican contingents. Note that the four contingents with highest 
GMAT quantitative means (Japan, People's Republic of China, Taiwan, and Korea) 
are among the five lowest contingents with respect to RVPI mean. In general, 
there are striking differences among the contingents in level of verbal 
performance relative to level of quantitative performance; and high 
quantitative means were obtained by contingents at all levels with respect to 
mean RVPI. Contingents also differ with respect to sex composition, 
proportion with U.S. undergraduate origin, mean year of birth, and other study 
varia bles. Contingents higher on RVPI tend to be higher on TCEFLEVL and 
TCEFLTCT as well as GHAT verbal. 



Preliminary Analyses of School-level Data 

As a preliminary step, sumary statistics (means, standard deviations, 
and missing data intercorrelations) for the variables described above plus the 
criterion variable (FYA) and the undergraduate GPA (UGBA) were computed, by 
school, for foreign ESL examinees only (a) to provide a basis for wi thin- 
school standardization, and (b) to permit assessment of the level of simple 
GMAT/FYA and other correlations, especially TOEFLICT/HA and UGPA/FYA, in 
foreign-LSL samples that were heterogeneous with re&pr.ct to all background 
variables. 
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TABLE 3 

PROFILE OF MEANS ON INDEPENDENT VARIABLES, BY COUNTRY OF CITIZENSHIP 



COUNTRY 


N 


GMAT-V 


GMAT-Q 


GHAT-T 


CANADA 


57 


36.366 


37.053 


599.947 


GREAT BRITAIN 


40 


34.600 


37.625 


* 92. 250 


SOUTH AFRICA 


10 


32.200 


33.600 


549.000 


PHILIPPINES 


36 


30.632 


31.053 


519.211 


PAKISTAN 


30 


26.300 


30.333 


*99.767 


JAMAICA 


12 


26.250 


27.063 


464.563 


ITALY 


13 


30.646 


35.536 


547.492 


NORWAY 


15 


26.933 


33.200 


516.667 


ARGENTIA 


13 


29.769 


35.536 


547.615 


FED. RP. GERMANY 


22 


27.500 


32.773 


506.616 


INDIA 


209 


29.435 


36.541 


543.301 


VENEZUELA 


22 


2? 664 


27.136 


444.616 


ISRAEL 


12 


30.417 


39.000 


563.563 


BRAZIL 


21 


27.190 


33.333 


496.333 


SWEOEN 


13 


26.692 


32.692 


501.231 


SINGAPORE 


25 


29.120 


37.920 


556.400 


SPAIN 


10 


26.500 


33.60C 


503.000 


MALAYSIA 


65 


24.369 


30.106 


467.092 


COLOMBIA 


26 


23.306 


26.731 


446.077 


FRANCE 


64 


27.613 


36.797 


531.453 


LE DAMON 


15 


26.200 


34.533 


501.933 


NIGERIA 


50 


19.460 


23.000 


366.440 


NETHERLANDS 


26 


25.692 


34.306 


501.923 


TURKEY 


13 


27.231 


37.077 


526.365 


GREECE 


35 


23.714 


32.514 


474.743 


HONG KONG 


77 


25.597 


36.461 


514.610 


MEXICO 


79 


22.709 


32.392 


463.747 


BELGIUM 


39 


22 .672 


33.5'fO 


474.672 


CHILE 


21 


23.571 


34.9C5 


404.905 


IPAN 


20 


22. 100 


32.6F 


466.050 


PERU 


19 


20.642 


33.26* 


459.947 


KOPEA 


146 


23.301 


42.027 


527 7AA 

7t f . fed 


TAIWAN 


217 


21.567 


40.525 


503.2*8 


THAILAND 


63 


16.819 


32.193 


419.422 


P. P. OF CHINA 


16 


21.167 


40.111 


496.667 


JAPAN 


156 


21.164 


41 t 


507.133 


OTHER COUNTRIES 


191 


24.178 


30.010 


464.613 


ALL COUNTRIES 


1924 


24.894 


35.399 


501.133 


U.S. f 1980-611 156684 


28.29 


26.79 


478.14 



vsscortp 

<Q*.6V) 

59.674 
56.705 
52.920 
49.432 
47.313 
42.833 
54.046 
50.560 
53.400 
49.273 
54.202 
41.455 
57.250 
49.646 
46.706 
55. 392 
49.5C0 
44.729 
42.715 
53.464 
50.253 
34.676 
49.723 
53.4E5 
46.743 
51.639 
45.473 
47.313 
49. 04 6 
46.110 
45.766 
56.006 
S3. 465 
42.284 
52.811 
53.742 

44.517 

50.314 

43.764 



SEX BIRTH YR 
<M=1/F=2I 



USUG=1 RVPINO^X TOEFMN T0EFTOT YESTOEF 



1.261 
1.150 
1.100 
1.342 
1.100 
1.500 
1.000 
1.133 
1.077 
.162 
.105 
.162 
.167 
.046 
1.154 
1.200 
1.000 
1.277 
1.115 
1.094 
1.000 
1.060 
1.115 
1.306 
1.U4 
1.247 
1.051 
1.000 
1.095 
1.400 
1.053 
1.041 
1.452 
1.434 
1.333 
1.064 

1.152 



56.649 
55.600 
55.100 
56.421 
56.133 
53.563 
56.692 
56.467 
55.000 
56.662 
56.952 
54.409 
53.167 
55.571 
55.769 
55.320 
57.900 
55.965 
55.423 
57.094 
57.467 
53.540 
57.269 
57.462 
56.429 
57.403 
56.401 
56.462 
56.476 
56.900 
56.156 
54.027 
55.613 
56.807 
54.S00 
53.766 

55.047 



1.161 55.616 
1.372 N A. 



0.353 

0.176 

0.0 

0.061 

0 667 

0.667 

0.091 

0.467 

0.063 

0.227 

0.095 

0.316 

0.200 

0.200 

0.091 

0.261 

0.300 

0.625 

0.346 

0.054 

0.417 

0.600 

0.200 

0.462 

0.246 

0.740 

0.056 

0.051 

0.053 

0.750 

0.316 

0.079 

0.093 

0.157 

0.056 

0.076 

0.361 

0.236 

N.A. 



53.370 
50.454 
50.126 
49.927 
47.120 
46.795 
46.566 
45.697 
45.001 
43.960 
43.694 
43.277 
43.111 
43.050 
42.650 
42.107 
41.627 
41.562 
41.163 
41.122 
40.626 
40.250 
40.072 
40.046 
36.659 
36.155 
37.740 
36.552 
36.495 
36.034 
33.664 
30.274 
26.977 
28.663 
26.733 
28.005 

41.383 

36.035 

50.000 



576.000 
540.000 
616.000 
594.000 
524.000 
567.003 
552.000 
576.000 
552.000 
563.000 
556.000 
493.000 
543.000 
515.000 
594.000 
567.000 
549.000 
559.000 
511.000 
570.000 
501.000 
553.000 
401.000 
510.000 
514.000 
505.000 
521.600 
565.000 
524.000 
456.000 
510.000 
513.000 
514.000 
472.000 
» 

504.000 
493.000 
493.000 



640. 0C0 
563.400 
612.750 
642.241 
610.375 

0.0 
604.000 
613.776 
600.633 
60*. 917 
421.164 
576.923 
595.000 
593.750 
622.091 
635.923 
574.000 
599.400 
563.526 
603.635 
607.633 
587.5ft- 
601.222 
560.000 
564.333 
579.929 
575.443 
572.135 
562.167 
535.200 
594.273 
576.265 
554.149 
543.157 
560.667 
581.692 

590.470 

565.234 



0.035 

0.125 

0.400 

0.763 

*.C67 

0.0 

1.000 

0.600 

0.923 

0.545 

0.699 

0.591 

0.333 

0.762 

0.646 

0.520 

0 600 

0.462 

0.731 

0.613 

0.400 

0.240 

0.492 

0.306 

0.666 

0.364 

0.666 

0.949 

0.657 

0.250 

0.579 

0.801 

0.634 

0.614 

0.833 

0.842 

0.435 

0.633 



DATA TAPLEO TOR COUM7RIE" WITH N=10f ONLY. 
FNTPU5 ARE NOT APPLICADIE. 



DATA FOR BIRTH-YEAR AND UNDERGRADUATE ORIGIN NOT AVAILABLE FOR U. S. 9 AND iQFfJ, 
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•DATA NOT AVDAILAftLE TOR REPRESENTATIVE SAMPLE OF (VAOUA1 E- SCHOOL POUNO TOFF 1,-TAK EPS. FIGURE FOR TAIWAN MAY PROVIDE REASONABLE 
ESTIMATE Or CENTRAL LEVEL tCF. GMJI SCORES A NO IOEJJ, TOTAL I. ENTRIES IN WE IfStOFF COUIfN INDICATE THE PROPORTION OF SlUOENTS 
FROM A COUNTRY MITII TOEFL SCORES AND PERMIT INFERENCES REGARDING THE NUMDER OF CASES USED 10 COMPUTE i|IE JOEFL TOTAL MEANS. 
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Findings of the preliminary analyses are eummrized below.* 

(a) The ssdian correlation between GttT quantitative scores and first 
year GA (r - .30) was the sese as tlet observed for the 85 general sasples of 
first-year students studied by the GMAC VSS during 1978-79 through 1980-81. 

(b) The ssdian correlation of ma verbal scores with FA (r - .16) was 
lower than that for the 85 general sasples (r - .25). 

(c) The ssdian correlation of VUSOOff with PA was .30; the median 
coefficient for GHAT Total with rA was r - .27. The loner ssdian for GMKT 
Total than for VSSQOV (Q + .6V) *sy be u nd e r s tood east siaply in teres of the 
lower ssdian validity for GHNT verbal then for quantitative and the fact 
that GAT contains sore verbal itsss than quantitative itess by a ratio of 
approximately 3 to 2. Thus, the less valid pred ictor (verbal) is weighted, 
urwe heavily in GAT Total than in VSS GQap. 

(d) Mian sasples were grouped acc ord i n g to siee (N< 20, N - 20-29, and 
N - 30+) , the ssdian GMMVO/FA correlatione (but not GMMW/FA or tcbfvfa 
correlations) varied inversely; across sasfie-siae categories (r - .39 for 
snaller, r - .30 for medium, and r - .25 for larger sasples). GAT-V/FA 
correlations did not very eysteeatically with sanple siee (r - .25, .07, and 
.19 for snaller, aediue, and larger sasples). larger seeples were found to be 
more higf \y selected on GHNT quantitative ability than the eediue or snaller 
sasples (see supplessntary figures in Appendix B). 



(e) Twenty-five schools in the study had previously subsitted data for 
. general student sasples to the GHftC VSS. Comparison of seane for the earlier 

"J "all student" (principally U.S. citizen) sssples from these schools with those 

O of their foreign EST udents in tha current study Appendix B.3) indicated 

that the quantitative esens of the foreign BSL students typically were higher, 
and the average verbal scores were sosewbst lower, than those for the student 
body generally. 



(f) The median TOEFL/FA correlation (r - .22), based on TOEFL-takers 
only, was slightly higher than the median GMAT-V/FA correlation (r - .16) 
which was based on all foreign BSL students in the respective school sasples. 

(g) For 21 schools supplying USE*, the ssdian correlation between USA 
and FA (r -.12) was lower than the GHM^VSS 85-school ssdian (.24.). 

These aedian coefficients reflect trends in the (reparative validity of 
the several predictors treated separately in school-level sasples of foreign 
ESL students: correlations for verbal predictors lower than those typically 
reported for general sasples; for GHKT qualitative, correlations with FA that 
are nore comparable to nose for sasples of U.S. students; for USA, lower for 
foreign students, stemming undoubtedly from their diverse educational origins. 

«lhe findings sumnarized briefly in this section are reported in detail in a 
report prepared for distribution to participating schools. The report is 
attached as Appendix B, which includes sane supplementary findings as well. 
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Jnalyses Based an Fooled mthin-Scoool Data 

m^m^^SS to P"* 1 * *ta for U.S. citizens. 
Within each school, original scons on the continuous variables were subjected 
to a s-scale transfozmtlen wim i*t*m*mr* *%r ami. •» .^jIj i.. 



s-scale transforation using pirrwters for the foreion ESL stodnts 
only-that is, tte original s£re? wsi^Sm* as 3» ^?25 



school *4th Bsans of sero and standard iswIspF oTKO. , » facilitate 
reporting, the s-scaled distribution* mm ttan T-acaled to foot distributions 
by school with aeans of 50 and etadsWdsvtatism M 10, AocoBdtoSTaU 
T-scal*d variables in the total a* \m^S^^m^!ltWt 

sintj the foreign EEL ptraastecs. Ihe seen T-scalad (s scaled! scores of 

%5 Sxm *' V*™***' indicate the average within-school 
deviations of their scores froa the aeans of foreign ESL exadnets, the 
general , Pfulation of interest.* fooled within-school data for various 
classifications of students were employed to test the following hypotheses: 

1) ORT/FYA correlations will he scderated by level of English profici- 
fncy--that is, the correlations batmen OMMF scores and FA will tend to be 
i ^ r i2*?S p L-?_f 0Wi ' 1 «tudants characterised by higher average 
if!Si?i , 5^ profid ^ tl,Bn "^groups characterised by lower levels 
of English proficiency as reflected by: 

a) epl vs ESL status 

b) Higher vs lower scores on TOEFL Total (TOEFL scale) 

c) Higher vs lower standing on the RVPI (original scale) 

d) Higher vs lower TOEFLEVL scores (original scale). 

2) The correlation between GMtt scores and FA will tend to be higher in 
a*groups that are hoeogeneous with respect to country of origin and/or 
assoc ^ t ?T "^S™"? variables than in subgroups that are heterogeneous with 
respect to these variables. 



•There nay be differences by school in the degree of representativeness of the 
foreign ESL sanples with respect to national origin and associated background 
variables, with corresponding effects n the aeans and standard deviations of 
the predictors. The average within-school standing of students from different 
countries on a given s-scaled predictor is not likely to correspond exactly 
with taair average standing in term of original scores on the predictor. 
This fact limits inferences regarding comparative performance of subgroups on 
the predictors, based on the s-scaled variables. For the data ofthisstudy 
there is a high degree of correspondence between the means of students by 
country on the original snd s-scaled variables. For example, for 17 analysis 
groups, largely h omogeneous with respspect to country, the rank-order 
correlation between average withir«-school standing and average standing on 
original OttT-Q scores was rho - .86; for the RVPI index, the corresponding 
relationship was rho » .97. 
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3) The correlation be ta — a OGBft, and PA will tend to be higher in 
subgroups of atud a nb i with 0.8. undergraduate origint than fix otfaar students. 



Regression analyses designed to assess the potential utility of TOEFL 
scores and the TOEFLEVL index as supplemental predictors of FA for foreign 
ESL students were also ccapletad. 

~?!??^* ^^F^****** of distributions of school-level 

coefficients indicated that Median OH^m correlations were loner for 

-due, by Inference, to ffiffen e nM a l degrees of restriction of range on GHsT-Q 
as a result of prianry aelerticn. Jteordingly, whatever feasiblav amines 
were replicated In subgroups of students defined in teres of the las oT 
samples of which they wen aeabers in order to introduce a asasure of control 
for the range-^estricticn effects. 

More detailed discussion of grouping criteria, analytic procedures 
employed, and related Batters is provided below. 

EEL vs ESL Status as a Moderator Variable 

For the analyses euaearised in Table 4, students were classified accord- 
ing To efl/esl status, and by sise of school-level saople. Larger schools 
were defined as these represented in the seaple by 36 or eon students; aadiua 
size schools were those repreeanted by 22-35 students and caaller schools were 
those represented by less than 22 foreign-ESL students, The asdians of the 
distributions of school-level OMH^PA coefficients for foreign-ESL students 
classified by sise of sample ware somewhat higher then the pooled within- 
school values shown in Table 4. zn evaluating this it should be noted that the 
sanple-size classification criteria were not identical. Mare important ia the 
fact that the schcol-level coefficients ware based on saaples differing 
considerably in size and the aedian is not sensitive to differences in the 
size of saaples. However, the pooled wi thin-school coefficients are exact 
equivalents of the weighted averages of the schcol-level saaples involved. 
Given the typically lower QKH^FYA correlations in larger than in smaller 
'icnool-level saaples, the weighted averages of the school-level coefficients 
would be expected to be saaller than the medians of the corresponding 
distributions of school-level coefficients. 

o OftT/FYA coefficients ware higher for EPL than for ESL students; in the 
EPL sample, without regard to school size, the V/TO and Q/HA 
coefficients and the V,Q/FA multiple correlation ware quite comparable 
to medians for the 85 GMRC VSS studies involving primarily U.S. 
students. 



o For ESL students, the pooled wi thin-school coefficients for GMAT 
verbal, quantitative, and combined scores, without regard to school- 
size category, ware lower than the corresponding 85-school medians. 
Across the three sample-size categories the correlation between GMAT-Q 
end fva was lower in the larger, more highly selected samples than in 
the smaller, less highly selected samples. 



31 



-17- 



Tfcble 4 



Pooled Sample Correlations of Selected Variables with FYA, by 
EPL/ESL Status and Size of School-level Samples 



Grouping 
Vdiiaoxes 


(N) 


GMAT-V 


GMAT-Q 


V,Q 


U.S.UG* 


SEX* 


YEAR OF 
BIRTH" 






r 


r 


R 


r 


i 


r 


EPL sample** 


157 


.255 


.326 


(.362) 


-.349 


.030 


.073 


Larger sch. 
Medium sch. 
Smaller sch. 


86 
37 
34 


.133 
.481 
.318 


.418 
.406 
.154 


(.419) 
(.537) 
(.319) 


-.295 
-.351 
-.458 


.011 
.282 
-.043 


.075 
.086 
.105 


ESL sample 


1762 


.180 


.239 


(.289) 


-.066 


-. 030 


.050 


Larger sch. 
Medium sch. 
Smaller sch. 


945 
552 
265 


.204 
.136 
.182 


.183 
.290 
.332 


(.265) 
(.314) 
(.365) 


-.029 
-.068 
-.184 


-. 084 
.039 
-.015 


.096 
.017 
-.043 



Note: V,Q is the best weighted composite of V and Q; the coefficient reported 
is the multiple correlation coefficient. For 85 general first-year MBA 
samples studied by GMAC VSS at ETS the median V,Q multiple correlation was 
•35; medians were #25 and .30, for V and Q, respectively. Larger schools were 
defined as those represented in the sample by 36 or more foreign ESL students; 
medium schools were those with 22 * 35 students; smaller schools were those 
with fewer than 22 foreign ESL students in the study. 

♦Negative coefficients for U.S.UG indicate mean FYA lower for U.S. 
undergraduate origins than for others; for SEX, positive coefficients indicate 
higher FYA means for women than for men, negative coefficients indicate the 
opposite; positive coefficients for BIRTH YEAR indicate a tendency for younger 
students to earn higher FYA than older students. 

**T-scaled means were 53.1 (FYA), 61.2 (GMAT-V), 49.1 (GMAT-Q). Means for 
ESL students, by definition, were 50.0 and standard deviations 10. 



ERLC 



BEST COPV AVAILABLE 



-16- 



Such a pattern of invent covariation of size of coefficients with sanple 
size it not evident in the data for the eeali samples of BL students. U.S. 
undergraduates had somewhat lower average Ha in both the BL and the SSL 
samples, but the relationship was stronger in the BL staple. Sen tendency 
for younger students to cam higher grades then their older classnates is 
evident for both staples, but no consistent direction is indicated the slight 
sex differences in Ha aeans. lhe coefficients for these variables vs FA are 
shown here prisarily to permit assesaaant of age, sex, and undergraduate 
origin (U.8. vs. other) as correlates of HA. 

There is no a priori reason to expect a consistent pattern of association 
(e.g., negative or positive) between these variables and first-year perform- 
ance across schools such as that which, on both theoretical and empirical 
grounds, is expected to obtain between GMNT scores end Tfc, None of these 
particular personal or background variables was found to add significantly to 
the wltiple correlation when stepped into a battery that included Gmx scores. 



Analyses of TOEFL and WH as Potential Moderator variables 

It is reasonable to believe that the GNAT scores correlate sore highly 
with fa for BL than for esl rtudents because BL students and U.S. citisens 
share similar linguistic, cultural, and educational heritages* lhe validity 
of GHKT scores of both BL and U.S. test-takers is unaffected by English-pro- 
ficiency related factors whereas the validity of scores for SSL students is 
likely to be lowered, invalidly, to sons extent by factors associated with 
their diverse backgrounds, especially differences in English proficiency. 

lhe potential value of TOEFL Total score (T-T) and the Relative verbal 
Performance mdax (RVPI) as moderators of QMT/FA relationships rests on the 
assumption that test validities for foreign ESL students classified according 
to score levels on these measures will tend to vary in such the saw way as 
those observed for EPL vs ESL status. 

For both the T-T and the SVPI analyses, classification according to level 
was accomplished by identifying scores which in a normal distribution would 
delineate the upper, middle, and loner thirds of the distribution. For TOEFL 
Total the sample values demarcating the classifications were 603 plus, 
567-602, and < 567 for higher, medium, and lower proficiency categories. Very 
few students scored below 500 (see plot of TOEFL total vs GKAff verbal scores 
in Appendix C). 

Examinees with a TOEFL total score of 603 are at approximately the 93rd 
percentile in the distribution of scores for U.S. gnduate-schcol-bound 
TOEFL-takers. and the corresponding percentile for a score of 567 is approxi- 
mately the 82nd; native English speakers tend to average above 600 on TOEFL 
(BIS 1981). Thus the average level of measured English proficiency in this 
ESL sample is high, relative to the average for all U.S.^aduate-school-bound 
TOEFL examinees. Considerable prior screening for English proficiency has 
taken place. 
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TDEFL scores were available for 1,205 students, erd missing for 559. it 
sews reasonable to assuae that the students without scores were screened for 



English ^proficiency by other seans. for example, a relatively strong 
association obtained in this sample between U.S. undergraduate origin and the 

^HHSL^JL !°5 L •222' J -U 99 M " 31 SF that U.S. undergraduates may have been 
exeapted from taking TOOL in asny instances. 

Bm A * i *f la 5 P"^ 08 * 1 ?? "*» «Ployed in classifying students according to the 
KVPI, available for all students. Higher, aediuaand lower scoring groups 

^JS 1 ?? ^ ft*! 00 ™,. * ^ «*< respectively. 

Students in the higher category have verbal scons less then one error of 
estimate below expectancy for U.S. CMKT examinees, those in the mediua-score 
category have verbal scores below expectancy by between roughly one and two 
errors of estimate, while those in the lei? category haveWrbal rores 
deviating from expectancy by roughly two cr more errors of estimate, based on 
data for U.S. 



Sfajdents in the respective English proficiency classifications were 
959*$°° *Y two sample-size categories— the medium and smaller samples for 
which data ware shown in Table 4 ware combined into a single smaller' school- 
sample classification. 

Pooled within-school correlation satri « were computed for all EST, stud- 
ents, and for larger and small sample-sis* classifications within the several 
proficiency groups, pa was regressed on QKS scores, and TDBFLTOT and 
TCmEVL^were added to assess their potential contribution as supplementary 

ft«n#*related finding*. Table 5 shews zero-order correlation coeffici- 
ent indicating the relationship of GMKT verbal and quantitative scores and 
total scores on TOEFL Total (T-T) and TQEFLEVL (T-L) for students in the three 
proficiency groups and for all students with T-T; coefficients are also shown 
for students without TOEFL. Means of the T-scaled variables are provided. 
These means indicate average relative within-school standing on the respective 
variables. Positive coefficients for T-L indicate a tendency toward higher 
phm for ^students from countries whose nationals have higher average scores on 
OTL than for ^those f rem countries with typically higher-scoring nationals. 
Students in the higher T-T classification had substantially higher within- 
school standing cn GMAT verbal than on GMKT quantitative, while for those in 
zHJXr m ^7^"? 1 ?* c ** groups, the opposite was true. standard 
deviations of the T-scaled variables (not shown in the table) were as follows 
for students in the higher, medium and lower T-T groups: verbal (9.2, 9.2, 
°;l f or « h i 9h 5 r i. ,n8diu !' lower groups); quantitative (10.1, 8.8, 9.9); T-L 
(10.1, 8.9, 7.8); and for T-T, the classificatory variable, (6.4, 8.7, 7.2). 

i £t¥ variable for a relatively strong systematic decrease in 

variability occurred across proficiency categories was T-L (TOEFLEVL). 

m Table 5, the underscoring indicates that for the designated predict- 
or (s), the observed correlations with pa increased steadily from lower to 
higher T-T classifications— e.g. , for the larger rohool group, successively 
higher (M-V/m coefficients (.114, .128, and .247) were found for lower, 
median, and higher T-T classifications. A consistent increase in GMAT/FYA 
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Tablt 5 

Means and correlations with FYA of Selected T-scaled variables, By Level 
of TOEFL score and Sample-size Category 



Means of T-scaled variables 



Group/ 
Sample size 


(N) 


FYA 


V 


Q 


T-L* 


No TOEFL 












Larger 

Smaller 

All 


(247) 
(312) 
(559) 


50.'. 
49.4 
49.8 


53.3 
51.2 
51.1 


47.8 
47.6 
47.7 


50.1 
50.5 
50.3 


Higher T-T (603+) 










Larger 

Smaller 

All 


(275) 
(164) 
(439) 


51.5 
50.8 
51.3 


53.2 
55.3 
54.0 


49.7 
49.8 
49.8 


53.5 
55.9 
54.4 


Medium T-T (567-602) 










Larger 

Smaller 

All 


(239) 
(140) 
(379) 


49.0 
51.5 
49.9 


46.8 
47.8 
47.2 


52.5 
52.3 
52.5 


47.8 
48.4 
48.0 


Lower T-T (< 567) 










Larger 

Smaller 

All 


(184) 
(201) 
(385) 


48.3 
49.3 
48.8 


44.9 
45.3 
45.1 


50.1 
52.3 
51.2 


47.3 
45.6 
46.4 



All T-T levels 

Larger ( 915) 

Smaller ( 817) 

All (1203) 



58.8 
59.7 
59.1 



47.2 
49.6 
48.1 



40.5 
42. S 
41.4 



All means - 50.0 
All means - 50.0 
All means - 50.0 



Correlation with FYA 



T-L T-T 



.171 .163 .049 — 
.110 .341 .017 — 
.140 .267 .030 — 



.247 .298 .156 .159 
72B2 7307 TTI2 .100 
.254 7301 7137 .136 



.128 .176 .142 .100 
7073 7277 7375 .048 
.111 7213 7115 .099 



.114 .108 -.020 .198 
7TBB 7250 ToTJS .181 
.150 7177 -7012 .193 



.214 .196 .147 .205 
.189 .271 .082 .124 
.204 .227 .119 .172 



Note, coefficients underscored are those that increase steaiily from lower to 
higher T-T classifications for the corresponding groups. Thus, for example, 
in samples from larger schools GMAT-V/FYA correlations increased consistently 
lower to higher t-t classifications. 

* T-L (TOEFL Level—country means ascribed to citizens in the sample )> 
T-T (TOEFL Total score). T-T classification is in terms of the TOEFL score 
scale. 
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correlation from lower to higher T-T for every subgroup is evident for GMMV-Q 
and TQEFTZVL, which was not thouftt of as a predictor whose relationship with 
FY* should be moderated fay TOEFL Total. Ihe sharply reduced TOEnb/L/lYh 
relationship in the lower T-T subgroup is associated with a sharply reduced 
standard deviation for the predictor in that subgroup, due to selection that 
is incidental to direct selection of the subgroup on T35n*-by inference, 
students in the loser T-T classification tsnd to come disproportionately from 
countries characterized fay lower-scoring contingents of U«S. -rand students. 

For the 439 students in the Uglier T-T classification, the observed 
GMftT/m correlations, for both the verbal and the quantitative measures, , are 
ccwparahl e to typical coefficients in samples of ttS. students sbudted fa^ dAC 
VSS and are like tfaoee for the sasflls of , foreign PL enrlnese inthjs study 
(cf . Wile 4). tae results wagpxt th* b?pottiisir tNfc Ml oT%agii«h 
proficiency as indeseC ly OTL tends to moderate OKt/TtK nrtaMciwtrfps ? by 
inference, in Mich the same wqr and for the mk§ reeeons tfcftfc KLvs OL 
status, per se, moderates these neUrtontfrips. For the pr tesot easfrli, the 
effect is pronounced only for students wth very high TOSFL scores of 
approximately 600 or greater, a level attained by fewer than 10 percent of 
U.S. graduate-school-hound TOEFL examinees. 

Sons insight regarding the potential role of T-L and T-T as predictors of 
fya is provided by the patte rns of sero-order validity coefficients in Table 
5: in the total GMKP/roDfL staple, T-T and (HMW nave roughly cosparable 
validity, in the M^er *-T sasple, validity for CHMMT is greater, but in the 
loner T-T sample, T-T has greater criterion-related validity then GMKMT. ihe 
multiple regression results shown in Table 6 provide further evidence regard- 
ing this trend. Ttoo analyses were run, one (A) with V,Q as the basic verbal/ 
qua ntitat ive predictor set and the other (B) with M^Q as the basic set. In 
the TOEFVQAT saople without regard to T-T level, regression outcomes were 
strikingly similar in sets K and B; this was also true for the medium T-T 
classification. Weights for all predictors were significant and adding T-L 
and T-T led to a modest increase in the multiple correlation. 

However, in the higher T-T subgroup, the weight for GMKF-V was signifi- 
cant but not that for T-T, whereas in the lower T-T subgroup, T-T became the 
contributing verbal predictor and the weight for GMKP-V was insignificant. The 
wight for T-L was significant in all but the lower T4 subgroup; this result 
may be explicable in terms of incidental rang* restriction on T-L due to 
direct selectio n on T-T, the classificatory variable. In the saople of 
students without TOEFL scores, the V,Q/TYA coefficient was R « .287; T-L did 
not make a significant contribution to prediction. Ihe V,Q/ra multiple was 
slightly higher then that obtained in either the medium or lower T-T 
classification. 

From the patterns of verbal and quantitative means for the no T-T 
classification (Table 5), and the correlational results, it may be inferred 
that the no T-T group probably is somewhat below the higher T-T group, but 
higher than the other T-T groups, in average English proficiency. 

The foregoing findings suggest that in general samples of students who 
have been screened on both GMKF and TOEFL, these measures are likely to have 
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Table 6 

Supplemental contribution of T-L and T-T to Prediction of FYA 

Standard partial regression weight 

Group (N) (R) 

~Sffi$ S«T T^L" FT - 
V Q 

Higher T-T (439) 

V,Q (A)# .224 .276 (.374) 

V,Q,T-L .187 .300 .130 (.394) 

V,Q,T-L,T-T .162 .308 .119 .057* (.377) 

T-T,Q (B)# .322 .175 (.348) 

T-T,Q,T-L .338 .143 .130 (.373) 

Medium T-T (379) 

V,Q (A) .092*. .206 (.272) 

V,Q,T-L .054* .234 .143 (.290) 

V,Q,T-L,T-T .036* .235 .135 .054* (.291) 

T-T,Q (B) .214 .100 (.236) 

T-T, Q, T-L .239 .142 .066* (.273) 

V,Q (A) (385) .127 .158 (.217) 

V,Q,T-L .128 .156 -.005* (.217) 

V,Q,T-L,T-T .082* .142 -.017* .148 (.258) 

T-T,Q (B) .154 .172 (.246) 

T-T, Q, T-L .153 -.005* .246 (.246) 

All T-T 

V,Q (A) (1203) .193 .218 (.296) 

V,Q,T-L .150 .237 .111 (.313) 

V,Q,T-L,T-T .109 .242 .088 .090 (.321) 

T-T,Q (B) .238 .185 (.293) 

T-T,Q,T-L .255 .107 .142 (.308) 

# In Set A, GMAT-V is treated as the principal verbal measure, and in Set B, 
T-T (TOEFL total score) is treated as the principal verbal measure. 

♦Weight not significant, p > .05 
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generally on apa rahle criterion-related validity. Bower, tht relative 
validity of and TOEFL aay tend to vary inversely with level of Ihglish 

proficiency. In EEL samples with high levels of acquired proficiency in 
English, GHKP-V My tend to fat a voce « pa y chom et ric ally e fficient" maeure 
than TOEFL, whereas for for less proficient student*, TOEFL say be the more 
efficient nature. (See Wilson 1962c, pp. 11 - I S, fa r a discussion of this 
proposition in the context of data for the general TOEfX/SNKT population). 

WW as Moderator . Classification of students acoonttiog to hvpi resulted 
in the identification of three subgroup differing markedly in relative 
standinq or verbal and quantitative (feu* 7)* ^-scaled verbal asans varied 
inversely mx quantitative means varied directly with RVPI level. Both «• and 
T-T nsans varied directly with WPI level . Direct eelection m WK loads to 
incidental range restriction or the other variables; for higher, medium, and 
lower RVPI classifications without regard to school alee, standard deviations 
were as follows: verbal (9.4, 9.5, 9.7; quantitative (9.4, 9.5, 8.7); *-L 
(10.4, 9.9, 7.9; IMF (9.5, 9.4, 8.2). In evaluating the coefficients, it 
should be kept in aind that T-T scores were aissing for 559 of the 1762 
students included in the RVPI sasple. 

The correlations of GMKF and other predictors with for higher, 
medium, and lower RVPI classifications (little 7) are generally similar in 
pattern to those reported (Tfcbie 5) for ocqparahle T-T classifications: 
and T-VPXA correlations tended to increase from lower to higher 
RVPI, and T-T/RA correlations were scnewhat higher than GMKP-V/na correl- 
ations in the lower RVPI subgroup. However, there are some differences in 
results: 

o In the T-T analysis, both verbal and quantitative correlations were 
relatively high in the higher proficiency group, but were considerably 
lower in both the radium and lower proficiency groups; 

o In the RVPI analysis, GKP-Q/FXA correlations, and GMAT-V/FYA correl- 
ations to a lesser extent, were relatively high in both the higher and 
nediun RVPI classifications. 

Table 8 shows selected results of regression analyses designed to assess 
the supplemental contribution of T-L and T-T by RVPI level. Using missing data 
regression procedures in order to include T-T as a supplemental predictor, in 
analyses without regard to school-sanpie size, T-T mode a significant supple- 
mental contribution in the Rioter and Lower RVPI classifications and was found 
to have higher weight than GMKMferbal in these analyses; neither V nor T-T 
made a significant contribution to the equation tor the Medium RVPI students. 
The missing data regression-procedures employed involved an assumption that 
the patterns of relationships for students without TOEFL and those with TOEFL 
are similar. 

The overall pattern of differences in moderating results for analyses 
based on T-T levels (Table 5) and the analyses based on RVPI levels is high- 
lighted by the multiple correlation coefficients for V,Q, / FA in the respective 
analyses: in the T-T analyses V,Q/FYA multipes were .374, .233, and .217 for 
higher, medium, and lower proficiency groups, respectively; for tfie" RVPI 



38 



-24- 
Table 7 



Means and Correlations with ITOk of Selected T-Scaled Variables, By 
Score-level on the RVPI and Sample-Size Category 



T-scaled means 



Group (N) 


fa 


V 


Q 


TOEFL 
Level 


TOEFL 
Total* 


Higher RVPI (42+) 












Larger (332) 
Smaller (257) 
All (589) 


51.1 
50.0 
50.6 


58.6 
59.1 
58.8 


45.5 
44.9 
45.2 


54.1 
53.6 
53.9 


56.6 
56.4 
56.6 


Medium RVPI (32 - 41) 












Larger (306) 
Smaller (257) 
All (563) 


49.7 
50.0 
49.9 


49.1 
49.6 
49.3 


49.7 
49.7 
49.7 


49.5 
51.1 
50.2 


49.1 
50.3 
49.6 


Lower RVPI (< 32) 












Larger (307) 
Smaller (303) 
All sch (610) 


49.0 
50.0 
49.5 


41.6 
42.6 
42.1 


55.2 
54.6 
54.9 


46.0 
46.0 
46.0 


45.7 
45.6 
45.6 



All levels 

Larger 

Smaller 

All 



( 945) 
( 817) 
(1762) 



All means ■ 50.0 
All mean' ■ 50.0 
All means -50.0 



Correlation with FYA 
V Q T-L T-T* 



.235 .330 .101 .206 
7TJ7 7323 .081 .142 
.194 .328 7091 .182 



.222 .268 .102 .114 
7305 7375 . 051-. 022 
.263 .317 7050 .064 



.135 .104 .078 .220 
7T75 730I .035 .209 
.158 .199 7055 .214 



.204 .183 .119 .205 
.151 .304 .053 .124 
.180 .239 .088 .172 



Note . Coefficents underscored are those that increase steadily from lower 
to higher RVPI levels for the corresponding groups. Thus, for example, the 
GMAT-V/FYA correlation increases steadily from lower to higher RVPI in sample; 
from the larger schools. 

♦Correlations for T-T are based on smaller samples of TOEFL-takers within 
each group. By RVPI group, the "All students" percentages with TOEFL Total 
were 57.7 percent (Higher), 69.3 percent (Medium), and 77.5 percent (Lower). 
Classification was according to the Relative Verbal Performance Index as 
originally scaled. 
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Table 8 



Selected Results of Multiple Regression Analyses 
by RVPI Level 



RVPI level/ 

pdyic size 


(N) 


V,Q 
(R) 


Add 

T-L 
(R) 


Add 
T-T# 

(R) 


Higher RVPI (total) 


589 


.331 


.339 


.364* 


Larger 
Smaller 


332 
257 


.340 
.323 


.351 
.329 


.377* 
.351* 


Medium RVPI (total) 


563 


.323 


.334* 


.336(b) 


Larger 
Smaller 


306 
257 


.272 
.386 


.290 
.392 


.291 

.HI*(b) 


Lower RVPI (total) 


610 


.216 


.225 


.274* 


Larger 
Smaller 


307 
.303 


.145 
.306 


.161 
.311 


.238*(a) 
.312* 



Note: Underscoring indicates that the sum of weights for the two added 
predictors is greater than the weight for GMAT-Verbal. 

#T0EFL scores are missing for a number of individuals in each analysis 
(see note to preceding table). ««"y«s 

* Weight of added variable is significant, p < .05 

(a) Only the weight for T-T is significant. 

(b) Weight for T-T is negative. 
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analyses corresponding multiples were .331, .323, ad .216. Both the T-T and 
the RVPI analyses indicate that the uxjfclln-proflcieir.: related variables have 
some potential as ayrd omental predictors, particularly among subgroups with 
lower T-T or RVP1. 

The observed differences in results reflect, perhaps largely, the effects 
of the added data for students without TOOL, who typically bad higher rela- 
tive standing m OMMT than on OKP-Q (see Table 57 and would, accordingly, 
tend to be disproportionately concentrated in the hi*er and sadJua RVPI 
classifications. By inference, the higher HWX classification includes a 
large proportion of the higher t-t students (for i4wa ««Mrhas "noreal" 
predictive validity} and the higher and sediua «R groups would include a 
disproportionately high conoantration of the students without TOEFL scores 
(for whoa the validity of OWT scores though attenuated somewhat, is still 
nigher than typical for individuals in the lower T-T or WI categories). 

lb the extent that the foregoing is true, it eeeas reasonable to infej 
that if all students had OTP- scores, the overall patterns of noderating 
effects for TOEFL and RVPI would tend to be ccsparable. 

toefl&vl and Gauntry of CtHssnsMp as irderaf m Variables 

Result s of the foregoing analyses suggest that the classification of 
students by TOEFL sr wee or RVPX leads to substantial incidental sorting by 
country of dtissmhip— for exaaple, the dispersion of TOBFLEVL scores) 
decreased steadily across the higher, sediua, and lower proficiency groups, as 
did their correlations with FBL This is consistent with the fact that (a) 
tceflevl classifies students according to the performance on TOEFL (mean 
scores) of all U.S.-^raduate-school bound students froa their respective 
countries and (b) there are aodest positive correlations between tceflevl and 
TOEFLTCT (r - .424), an4 RVft ( .351). (See Exhibit A ar- *ahle 2) . 

It was expecter'. (a) tbit GMW/m correlations would tend to be higher 
for students with higher TOEPLEVEL scores than for students with lower 
toeflevl scores— that is, for students froa countries whose U.S.-bound nation- 
als typically have higher TOEFL swans than for students froa countries with 
lower-scoring student contingents. It was also expected (b) that GKAT/TYA 
correlations would tend to be higher in staples that are homogeneous with 
respect to country of origin than in samples that are heterogeneous with 
respect to this variable; moreover, to the extent that the hypothesis (a) is 
valid, it would be expected (c) that in sasples that are homogeneous with 
respect to country of origin QKT/FYh correlations would tend to be higher in 
sasples from countries with typically higher-scoring contingents than in 
countries with typically lower-scoring contingents. 

Evaluation of hypothesis (a) . Students were classified according tj 

TOEFLEVL (T-L) as either Higher (scores of 550 or greater) or Lower «550). 
The Higher category included primarily students (N - 643} from European coun- 
tries or countries in which English is an official language and/br an academic 
lingua franca at the level of higher education—for example, India, the Philip- 
pines, Malaysia, Nigeria, Singapore, the Caribbean, etc.; the lower classifi- 
cation included primarily students (N - 1,119, from Asian countries in which 



41 



there is note liadtcd exposure to English (e.g., Taiwan, Thailand, Korea, 
Japan) , and students f roe Mexico, Central and South Jewries, and tits Mideast. 
Predictor/m coefficients for students classified by TCBUEVL and by 
school-sasple sise are shown in Table 9. 

Consistent with expectation, s&riants in the higher T-L classification (N 
- 643), OW/FYA correlations ( WU - .200 r Mft » .368 , V,Q/rXK - .382) 
were systeeatically higher i±an coBrarabir~cx3etoaen5~( J9i t r ^Sd 
.232) for students in the Loner T-L daaalficetigB {» - l,119TT~0elngTsls»^ 
data pr ocedure s with the lieiting ass umption of similarity of the TC0L~taking 
art non-TOBPXHaldng subgroups, TOeYtflOTvO/HA mflrtples for the two TC0LEVL 
subgroups without regard to school ss e ple sis* (ooefficivots Dot shown in the 
table) were .382 and .231, respectively, ooererahle to those for v,Q in Table 
9; and it is clear froe Table 9 that adding *4 mAfal te> theV,Q ooeposite 
did not lead to a potentially practical incretent in the eultiple correlation, 
owr/m relAtionehips were low»r in the larger saaple si as category which 
included the sore highly selected seaples. 

This particular classification schene identifies a subgroup based on 
historical country-level data alone for which the QHO/nk eultiple correl- 
ation is relatively !dgh--ooBparable to the 85-school GMAC VSS aedian. Only 
about 36 percent of the total is in this subgroup. QtKS/Tik correlations for 
the remaining students are rather markedly lower . T he withir. -school standard 
deviati ons we re generally ccnrarable for the TCEFLEVL classifications. The 
higher TCEFEVL subgroup had relatively higher wi thin-school standing on verbal 
than on quantitative, whereas ths opposite was true for the lower TCOUNL 
subgroup. 

Valuation of h ypothes es (b) and (c) . Analyses of GMM/Ttt relation- 
ships were conducted in 23 subgroups, the majority of which included citizens 
of a given country only. In a few instances, students froa several countries 
that were judged to be similar in important respects were included in a given 
analysis group— for example, one group consisted of students from several 
Arabic-speaking, primarily Mideastem countries, another of students from a 
muter of European countries, and still another included data for (largely) 
Spanish-speaking students from Central and South American countries (see 
Exhibit A for detail regarding the countries included in analysts groups that 
were heterogeneous with respect to country of citizenship). 

Pooled, wi thin-school correlations 'OAT/FA and TOEFL/FYA), based on 
T-scaled variables, are shown in Table 10 for the respective analysis groups. 
Analysis groups narked by a double asterisk are those characterized by 
typically higher-scoring TOEFL contingents (TOEFLEVL - 550+) t others tend to 
have contingents scoring below 550 (see Exhibit A). Means and standard 
deviations of raw and T-scaled scores (the latter reflecting relative standing 
within school) on all study variables for these analysis groups are provided 
in Appendix C. 

Because of sample-size considerations, the VSSOCMP/FYA coefficient, 
rather than the multiple correlation coefficient, is shown to reflect the 
joint relationship of V and Q to the criterion— VSSOCHP is a standard 
co i.osite (0 ♦ .6 V), reflecting the ratio of the average of optimal » «'*hts 



42 




-28- 
Table 9 

Selected Predictor/m correlations for Students aassified 

by TOBFLEVL 



TOEFLSVV 
Sample 


(N) 


GMftX 
Verbal 


GHAT 
Quant 


T-L 


T-T 


V,Q 


Add 
T-L 


Add 

T-T 






r 


r 


r 


r 


R 


R 


R 


Higher T-L 


643 


.200 


.368 


-.022 


.124 


.382 


.382 


.387 


Larger 
Smaller 


322 
321 


.184 
.207 


.337 
.401 


-.049 
-.014 


.180 
.042 


.353 
.410 


.353 
.411 


.378* 
.414 


Lower T-L 


1119 


.134 


.194 


.040 


.126 


.232 


.234 


.246* 


Larger 
Smaller 


623 
496 


.160 
.101 


.143 
.258 


.036 
.046 


.133 
.124 


.207 
.280 


.210 
.281 


.220 
.298* 



Note: TOEFL (T-D scores are missing for a number of students. 
Higher T-L - 550+ ; Lower T-L - <550 

* Weight for added variable significant, p <.05 
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Ifcble 10 

Correlation of Designated Predictors or Composite Predictors 
with FYA, Based on T-*ealed Wi thin-school Data: 
By Analysis Group 



Analysis 


( N ) 


GHAT 


GHAT 


VSS 


TOEFL Tot 


group* 




Verbal 


Quant 


Coup 








k 


r 


r 


r 


x 


(N) 


01 Mideast 


61 


.137 


.338 


.379 


.045 


27 
x / 


02 Thailand 


83 


-.018 


.125 


.099 


.203 


51 


03 Taiwan 


216 


.018 


.149 


.141 


.164 


181 


04 Korea 


146 


.156 


.251 


.282 


.007 


117 


05 Jaoan 


1 5ft 

J. JO 


171 
• 1/1 


. ibi 


.30/ 


.228 


133 


06 Hon? Knni? 


77 




— • 0J7 


-•013 


.045 


28 


02-07 




1 in 
• 11U 




.186 


.152 


510 


08 Mexico 


79 


.139 


.278 


.268 


.203 


70 


09 S.America 


147 


.123 


.289 


.276 


.030 


103 


08-09 


226 


.144 


.290 


.283 


.122 


173 


11 Greece-Turkey 


55 


.080 


.155 


.229 


.112 


3i 


12 Pakistan 


29 


.099 


.294 


.315 


.014 


6 


13 Malaysia** 


64 


.052 


.291 


.288 


.277 


30 


14 India** 






• 416 


.406 


.074 


144 


IS Nigeria** 


44 


.167 


.434 


.427 


.454 


11 


16 Singapore** 


18 


-.083 


.433 


389 


-.128 


10 


17 PhilinninM** 


J/ 


• »/3/ 


.351 


.359 


.327 


28 


12-17** 


396 


.19^ 


.387 


.388 


.130 


231 


19 France** 


64 


.181 


.419 


.407 


-.017 


52 


20 Other Europe** 


164 


.131 


.286 


.268 


.133 


126 


19-20** 


228 


.141 


.320 


.302 


.098 


178 


22 Other 550+** 


42 


.431 


.277 


.381 


.532 


16 


23 Other < 550 


74 


\ 


.411 


.506 


.312 


36 


Total ESL 


1762 


.180 


.239 


.284 


.172 


1202 


Total EPL 


157 


.252 


.326 


N.A, 


Not applicabl 



* Analysis groups are listed in generally ascending order with respec to 
TOEFL Level. See Exhibit A for TOEFLEVL (TOEFL means) for the countries in 
the respective analysis groups. Group 09 includes Central as well as South 
American countries; Group 11 includes Cyprus; Groups 22 and 23 are 
classifications based entirely on TOEFLEVL (550 or above, or less than 550) 
for countries not elsewhere clarsified. 

**Countries whose U.S.-graduate-school-rbound nationals typically score 550 
or higher on TOEFL. 
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f or v and Q in general samples of at* dents baaed on studies previout-'y 
conducted by the CMC VSS for 25 of the schools participating in the present 
study, In several analysis groups, the standard composite of verbal and 
quantitative scores vQ + .6V) was lass closely related to fxa than 9KP-Q 
indicative of the disparity betMSen the (lower) V/Kkmd the (higher 
coefficient. Cbefficienta for OM Tptal, not shorn in the table; tare also 
typically lower than that for (MO-Q alone. There are sore verbal than 
quantitative itaaa in the GMKf. Thus, the Okr total soon gives sore weight 
to the verbal items, which tend to have lower validity in these samples, than 
to quantitative itaaa, which tend to have higher validity. Similarly, VSSOQHP 
nay tend to give too such weight to the verbal coeponent, TOOL-Total/FXA 
coefficients and the nusber of cases on which they are based are also shewn in 
Table 10. 

o GMKP-Q/FYA correlations by analysis group were higher than the corre- 
sponding total ESL-saaple coefficient (r - .239) in .all analysis 
groups except those ooapoaad of students from Thailand, Taiwan, Hong 
Kong, and <keece-Turkey<yprus; coefficients for the three Asian 
contingents were especially low. 

o m all but four of the analysis groups, oaow/m correlationa were 
lower than that foe the total EBL sample (r - .180); and with few 
exceptions, the dM^Wft correlations wars higher than the 
corresponding within-echool coefficient for all EBL students. 

Results for several coabinad analysis groups (groups 02 through 06, 08 
through 09, 12 through 17, and 19 through 20) shorn in the table, Indicate 
that GMAT-Q/FYA coefficients wire higher for students from countries with 
typically higher-scorin g TOP L^akers, than for students from countries with 
typically lower-scoring TOEFL-takere— for the respective suaaary groups, Q/rtk 
coefficients ware .154, .290, .387, and .320; the corresponding OvYT-v/m 
coefficients wire .110, .144, .199, and .141. Thus, the pooled within-school 
GMKT-Q/FYA coefficients ware higher for students f roe European countries and 
from the several Asian countries in which English is an iaportant academic 
language than for students from Mexico, Central and South America, Thailand, 
Taiwan, Korea, Japan, and Bong Kong. 

Findings for the several coabined analysis groups are consistent with 
findings reported above to " higher and lower TCEFLEVL classifications uat did 
not break out data by country. According to Hypothesis C, there should be a 
relatively clear tendency for the OfoVFYA relationships to be higher within 
the respective TDEFLEVL classifications whan country ia controlled than when 
data are analysed without regard to country. Such a tendency ia not clearly 
evident in Table 10. For example, QMP-q/iya coefficients for analysis groups 
conposad of the Higher T0BLEVL students (those marked by double asterisks) 
are r oughly comparable to that reported earlier (Table 9) for the Higher 
TOEFLEvL classification of students— no systematic enhancement of the 
GMKP-V/FYA relationship due to control over country ia evident for these 
analysis groups. However, except for the coabined Asian samples (02 through 
06), coefficients for other combined analysis groups were higher than those 
(V/m - .134, Q/m - .194) reported in Table 9 for students in the general 
toeflevl < 550 classification. 
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TWo major trends emerged in the findings involving moderating effects of 
the test and background variables. First, classification both by test 
measures of English p ro fici en c y and by country a pp eare d to have a moderating 
effect, especially en mm^fA relationships; sorting fay the test measures 
led to substantial incidental sorting by country, arcs vies versa. Second, 
control for country of citiwnship, per ee, had a moderating effect on Q/tYk 
correlations (these vert inmeihst higher than in the general wmf&*) 9 but 
v/m correlations, by ocujtry, name somertat lame than the c o crsqponding 
correlation in the total BL saapLe. Such a pattern, *hich use not expected, 
suggests that the pattern c£ correlations between Gttft meens aMmmeemfor 
students class ifie d by country is different from the pattern of within-school 
GMAT/rav correlations. Some related findings that shed li^ht on these two 
trends are presented below. 



Belated Findinrs 

ihe moderator analyses involving classification fay country did not take 
into account individual di fferences within countries on Boglish proficiency- 
related variables, and the TOEFLTOnL and BMPI analyses did not take country 
of citizenship into account. Some indication of the degree of incidental 
sorting on country that is involved in the classification of students accord- 
ing to the Bnglish-proficienCy related test measures is provided in Figure 1. 

The figure portrays graphically trends in the distributions of RVPI 
values for students in the analysis groups shorn in Tbble 10, ordered from 
lower ' ar (left to right) in terms of maan BMPI. Ihe vertical bars in 
the fit. epreseot the range of RVPI values included in the middle two- 
thirds of the original BVPl-score distribution of each contingent (not the 
T- -scaled within-school distribution); the horizontal lines correspond to the 
RVPI values that were used to classify students into higher, medium, and low- 
er RVPI subgroups for the analyses reported in Table 7. At the top of the 
vertical bar for each analysis group, v,Q/FYh correlations (the VSSOGMP/FA 
coe fficie nts from Table 10) are e ntere d; at the bottom of e ach ve rtical bar, 
the TOEFIZVL index value (mean TOEFL score of U.S. -bound TOEFL-takers) is 
shown. 

Ihe lower RVPI classification clearly includes a disproportionate nunber 
of students from Taiwan, Japan, Thailand, Korea, and the Mideast uhile the 
higher RVPI classification includes disproportionate numbers of students from 
countries uhere English is an official or academically important language, or 
countries. 

It may be seen that students from Hong Kong, uto were classified with 
contingents from Thailand, Tfciwan, Korea, and Japan in terms of TOEFLEVL, have 
substantially higher RVPI sec res than the other three contingents— by infer- 
ence, perhaps 75 percent of the Bong Kong accidents are in the medium and 
higher RVPI classifications, vhereas 50 percent or more of those from the 
other Asian contingents designated were in the Lower RVPI category, judging 
from their higher RVPI scores (which index higher GMKT verbal scores as well), 
and the fact that only 36 percent of the Bong Kong students presented TOEFL 
scores as coopered to over 80 percent of those in the other three contingents, 
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the average level of English proficiency of the Bong Kong sample pr esunably 
was higjber than that of the other Asian oontingtuts. 

In any event, the variations in observed QKT/FA correlations ascng the 
four Asian contingents (analysis groups 02-06) cannot logically be attributed 
to di ff e re nces in levels of Ehgllfh proficiency nor cm the relatively low 
within-school correlations far the Bona Rang sasple be explained solely in 

as well as English proficiency related considerations that sight tend to 
aifect the level of \JQ£T/F& correlations should be taken into account in 
evaluating the findings for these end other contingents of foreign ESL 
students* It ny be recalled, for exasple, that these Asian contingents 
averaged above the 90th percentile on Q9Ct-Q. 

This point is reinforced by the data in isble 11, tfiich shows pooled 
within-school Qna/m correlations for the four s*jor oc s bin e d analysis 
groups and for selected individual analysis groups, for students fras schools 
represented by larger, soce highly-eelected sasplee of foreign students and 
schools represented by asaller, less highly-a( Ejected sespHes, respectively. 
Note that for analysis groups 02, 03, «nd 06 ( Thailand, Ddwen, and Bong 
ifcng), GMwym coefficients tend to be higher in the asaller, less-selected 
schcol-samples than in the large, sore highly-selected sasplee. 

It may also be determined frcn Table U that disproportionately high num- 
bers of students frw analysis grant OS, 00-09, end 19-20,. were in the larger 
sanples while disproportionately higft nuabers of student fro* the other 
analysis groups were in the asaller sasplee. Again, QMT/FVK correlations nay 
be influenced by selection-related as well as Ehglisiv-proficiency related 
factors. 

Correlation off T-scaled OKI and FML As indicated above, control 

for country of citizenship, per se, resulted in Q/KA correlations that were 
somewhat higher, but V/FYA correlations that were soomhat lower, than the 
corresponding correlations in the total ESL sasple. Hiis unexpected finding 
suggested that there were di f ferences in the anong-groups ©wr/RA correla- 
tions for V/KA and Q/RA, respectively. More specifically, this result 
suggested the possibility of a higher degree of correlation between the mean 
T-scaled standing of the analysis groups on QIKP-V and FXA, than between the 
<2*flV-Q and FXA means of the groups. 

Figure 2 shows plots of T-ecaled nsans (fras Appendix C-l) for 17 analy- 
sis groups (all but the najor contained groups in Table 10) , on RA and desig- 
nated GMAI ^predictors: for caWP-V/FA (Plot A), QOT-Q/HA (Plot B), GMftT 
VSSOOKP/FHMPloT # , and Total/KA (Plot D). These plots indicate the degree 
of association beteen the average within-school standing of the respective 
groups on the designated predictors and their average standing in terns of 

FYA. 

In evaluating the observed differences in T-scaled FYA means, it is 
important to recall that the* a means reflect average deviations from school- 
level FYA means for selected saqples of foreign-ESL students* Although every 
school-level sample was heterogeneous with respect to analysis-group 
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Table U 

GMAT/FYA Correlations for Combined Analysis Groups, by Size of 

School-Sample 



Larger (more selected) Smaller (less selected) 

"nples samples 



Combined 


N 


GMAT- 


6MAT- 


VSS 


N 


GMAT- 


GMAT- 


VSS 


analysis 


grps* 


V 


Q 


COMP 




V 


Q 


COMP 


02-06 


352 


.11 


.10 


.13 


328 


.11 


.22 


.25 


02 


20 


-.33 


.08 




63 


.07 


.14 




03 


95 


.00 


-.02 




121 


.03 


.29 




04 


82 


.18 


.22 




64 


.11 


.30 




05 


121 


.16 


.24 




37 


.21 


.33 




06 


34 


.04 


-.20 




43 


.13 


.20 




08-09 


169 


.10 


.26 


.24 


57 


.27 


.37 


.42 


12-17 


141 


.13 


.31 


.30 


255 


.20 


.44 


.42 


19-20 


173 


.16 


.36 


.34 


55 


.08 


.21 


.20 



All ESL** 945 .20 .18 .25 817 .15 .30 .33 

* 02-06 (Thailand, Taiwan, Korea, Japan, Hong Kong); 08-09 (Mexico, Central 
and South American countries); 12-17 (Pakistan, Malaysia, India, Nigeria, 
Singapore, the Philippines); 19-20 (France, other European countries). 

** Ns are greater than sum of column entries since not all analysis groups 
are treated in the table. 
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Figure 2. Plots of T-scaled means on designated GMAT predictors and 
FYA for analysis groups (see Table 10) 
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nanbership, all the analysis groups wn not rtprsssntsd in every school 
Maple, and tht proportional representation of groups varied somewhat across 
schools. In tl» drousstanoM, sssll di fferenc e s in ssen FYA between groups 
should not be jafhasissd. Attention amy properly be fbcussed; however, on 
general trends in the pe rfb rasnot data, such as those portrayed in Figure 2. 

o Proa P.'ots A and B, it say be interred that the average within-school 
perfoiTjnoe (ssen T-scaled PSA) of the respective analysis groups 
termed to correspond sore closely with their average within-school 
(relative) standing on QMMferbal, than with their relative standing 
on GWUKfcantitative. 

o in Plots C and 0, it say be seen that ssen T-scaled GMKT Dotal tended 
to correspond sosswhat sore closely with ssen T-scaled PYA than did 
ssen T-scaled VS90QMP; verbal iteas are sore heavily weighted in GHAT 
Total than in VSSCCMP, so thU finding is consistent with the pattern 
of findings in Plots A and B. 

It appears (a) that individual differences in PA within the respective 
analysis groups are sore closely associated with GHAT quantitative than with 
GHAT verbal, but (b) that for analysis-group di f ferences, the opposite is true 
-mean differences in T-scaled HA were associated sore closely with dif- 
ferences in T-scaled verbal sssns than with di ff e r ence s in T-scaled quantita- 
tive means, lhe fact that the GHftT-V/ttA correlation was higher in the total 
ESL sample (heterogeneous with respect to country of origin) than in the 
respective analysis groups (relatively hosoe a n e ou s with respect to country) 
thus appears to be explained, statistically, by a relatively strong 
correlation b et wee n the T-scaled verbal and HA (criterion) aeans of the 
respective analysi s grou ps, m evaluating this result, it is useful to recall 
that the variable TXJOWJL, which was forsed by ascribing to students the 
TCB7L ssans of U.S.-graduats-school4»und TOEPL-takers fraa the respective 
countries, contains significant HA-related variance— it was positively 
correlated with PYA in nationally heterogeneous saaples. TOEFLEVL was thought 
of as reflecting differences in "richness of English language background" for 
students from different countries. 

This apparently anosalous pattern of results is understandable, arsuming 
the taxability of bV< following propositions: 

a) Differences ascng the analysis groups in average performance on the GBE 
verbal measure tend to reflect average differences in level of 
developed proficiency in English, as such as (in addition to) differ- 
ences in level of developed verbal reasoning ability, which the verbal 
test measures in saaples of U.S. studen' ?, This may tend to be true as 
well for individual differences in verbal test performance within the 
respective analysis groups. 

b) The differences in English proficiency that affect verbal test 
performance also affect academic performance. 

in developing this rationale, it was reasoned that within the respective 
NBA programs, which include students from different countries (analysis 
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groups), student performance if judged without rtgacd to their national 
origin, in carrying oat tht full canga of aoadaaic activities characteristic 
of the first year of study— involving within-classrooa interaction, perform- 
anca on periodic oxasdmtions written under time constraints similar to those 
lnco eed bv standard tests* and written issactises ts&side the classroom stu 

•■■•■■Bm^pT^mm^m^pB l^^lp^p^mBeMe^^B"mmT^B» ^BWWw , ^P"»lmW^P^e^P"JP% ^^p^a^s^mm^^B^Jp^mF^sw^swe^F ^^B^eF^e^*^ 'W^mmjBF ^ew>e»^BF ^^^Wf»^^^ ■mm F, w^» 

dents with high quantitative ability but lew English, proficiency art at a 
competitive disadvantage vis a vi* their gDuntarparts with .richer ; English 
language backgrounds eixi^gxaatar .foolish r esi de ncy , and their Relative 
academic productivity sufiars accordingly* ftus, Variants froa countries *hose 
U.S. bound nationals typically asd^t lower levals o^ English proficiency (as 
indexed by lower average scores 1 on both GMNE verbal and VM/L* for example), 
tend to receive lower F» in WK progress then stoosnts la the typically sore 
proficient groups (country contingsatsf even though the latter say have lower 
average quantitative ability. Put sisf% a Variable, like language say affect 
pcrforeance on both the predictor and the, criterion variable, leading to 
patterns of predictive relationships that cannot be explained in terns of the 
construct which the test is de si gned to measure. 

The foregoing line of reasoning also helps to explain both the low 
relationship betw een T-scaled saans of the analysis groups on QW quantitat- 
ive and FVk, and the finding that the OlHfA correlation in the total ESL 
sample (h eter ogen e ous with respect to national ccaposition) was lower than the 
corresponding coefficients within the respective analysis groups (country 
contingents of students with sisdlar English language backgrounds). Both of 
these findings appear babe explained ptinarily by the fact that, oh the 
average, the lYks of students in national contingents characterised by high 
quantitative ability, but low English proficiency, tended to be sore, consist- 
ent with their level of English proficiency (as indexed by their low verbal 
scores) than with their high average scores on the SWT quantitative measure. 
When data are analyzed by country (analysis group) this inconsistent 
predictor-criterion covariance is eliminated. 



Undergraduate Origin as Moderator of UGEA/WA. Relationships 

Undergraduate GPfc (USA) was provided for only 564 of 1,762 foreign ESL 
students from 22 of the participating schools. Students with UGBfc were 
classified according to undergraduate origin (U.S. vs other) arid school-sample 
size. About 28 percent had attended a U.S. school. Son 71 percent (402 of 
564) of all students with US* were in the larger, sore highly selected 
samples; 63 percent of those with U.S. undergraduate origin as compared to 74 
percent of those with international undergraduate origins were in the more 
selected samples. 

Consistent with logical expectation, UGEA/IYA correlations were much 
higher for students with U.S. origins than for those with diverse 
international origins. 

o In analyses involving data for 157 students with U.S. UGPAs, the 
UGPA/m coefficient was .262; the V,Q/m multiple was .180, and 
adding UGPA resulted in a muTHple correlation of r - .324. 
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o For 407 non-U.S. UOfcs, the corresponding coefficimnts were r - .013 
(UGPA/Ftt), R - .264 (V,Q/TO0, and R - .266 (adding U3PA). — 



This study was designed to explore the effect of selected test and 
background variables on the pooled within-echool relationship between GNAT 
scores and Kk, and to assess the pote n tial role of selected fOBFL-related 
variables as supplemental predictors of FA for foreign students. ■ 

OKP/m correlations vera food to be s cde ra te d by Eft vs ESL status, 
per se— for the snail sample, of students whose reported native language was 
English (N - 157), GMKT/fYA coefficients [V/rtK Om, and V,OmT were 
.255, .327, and .362; cceparable coefficients for the basic foreign SSL sample 
were .180, .239, and .289 (see Table 4). 

m the foreign-ESL ■ am pl e (N - 1,762) moderating effects on OKU/FA 
correlations were found when students were classified according to individual 
diffe rences on two English-proficiency related measures, namely, scores on 
TOEFL and the Relative Verbal Performance Index (PVPI). 

in analyses involving 1,203 foreign ESL students with scores oA TOOL, 
OKF-m coefficients (both GMKP-V and GMKP-Q) were relatively high 
(ccnparable to typical co eff icients observed in studies involving samples of 
U.S. MBA students) in the subgroup (N - 439) scoring 603 or higher, but not in 
two lower-ecoring subgroups (Table 5) . 

m regression analys es based on date for the GMKT/TOBFL sample, TOEFL 
Total score (T-T) and TOEfLEVL (T-L) were found to have significant weights 
when treated as additional predictors in a hattery that included GMKP-V and 
GMKP-Q 'Table 6). in the higher-scoring T-T sub-sample, the weight for GMKP-V 
surpassed that for TOEFL Total, but in the lower-scoring T-T subgroup, TOEFL 
Total rather than GMKP-V was the primary verbal predictor in the battery. 
When TOEFL Total was substituted for GMKP-V as the primary verbal predictor, 
multiple c orrela tions with the FYA criterion were quite comparable in the 
total GMNP/TCEFL sample, for students with lower T-T scores, the T-T,Q,T-L/FA 
multiple (R - .246) was higher than the V,Q,T-L multiple (R - .217). 

When students were classified according to RVPI level, GMKP/m coef- 
ficients were relatively high in the two higher RVPI-level classifications 
representing 1,152 of 1,762 ESL students (Table 7), and relatively low in the 
lower RVPI subgroup, trends consistent with hypothesis. 

In the higher and medium RVPI classifications, GMKP-V/FJA coefficients 
were somewhat lower than those observed for the higher-scorers on TOEFL or for 
the foreign EPL sample. Using missing-data regression procedures (with the 
limiting assumption of similarity of TCEFL-taking and non-TCEFL-taking 
stu dents) , findings (Table 8) regarding the supplemental contribution of TOEFL 
and TOEFLEVL were generally similar to those in the basic GMKT/TOEFL sample 
(including only students with both scores). 
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GMAT/FYA correlation* >«re found to bt moderated when students were 
classified according to TOEFLEVL, as higher (H-L - 550») or lower (T-L <550) . 
As hypothesised, for students in the higher TOEFLEVL classification (from 
countries whose TOEFL-taking nationals average 550 or higher), V, Q, and V,Q 
correlations with FA were higher (.200, .368, and .362, from Table 9) than 
the corresponding coefficients for students in the lower TOEFLEVL subgroup 
(.134, .194, and .232 foe V, Q,/*nd the V,Q cosposits, respectively). 

Students were classified into 23 analysis groups, the majority of which 
were h ce pg e n eous with respect to country of citisanship. The GMKP-Q/Fa. 
relationship was soderated this classification schass for all but four 
analysis group (Table 10). 

GMNT-Q/FYA correlations were higher for students f row European countries 
and from several countries in which English is an i mp or tan t academic language 
(India, the Philippines, Malaysia, for example), than for students from 
Mexico, Central and South America, and from Asian countries in which Bnglish 
is not a widely-used academic language (e.g., Taiwan, Taiwan, Korea, and 
Japan) . 

Contrary to expectation, GMKF-V/RA correlations for students classified 
by country tended to be lower than the corresponding coefficient in the total 
ESL sample (Table 10). This unexpected outcome appears to be accounted for, 
statistically, by a related finding (Figure 2), namely, that the T-scaled F3CA 
means of analysis groups were more closely asso ci a t ed with their T-scaled 
means on GMKT verbal (the less valid predictor) than with their T-scaled means 
on GMAT quantitative (the more valid predictor) . These results are understand- 
able if it is assumed that differences, among the analysis groups in average 
scores on GMKT verbal tend bo reflect group differences in English proficiency 
that affect both verbal test performance and per for mance in MBA programs. 

Analyses of moderating effects by TOEFLEVL and by country of citizenship 
did not take into account individual differences among students with respect 
to level of English proficiency as indexed by TOEFL Total scores or the HVPI. 
And, analyses of GMKT/KA correlations in subgroups defined in terms of the 
two test variable did not consider country of citizenship. However, sorting 
by courtry of citizenship results in substantial incidental sorting on the 
English-proficiency (test) variables, and vice versa (Figure 1). 

GMAT/m coefficients were especially attenuated in samples of students 
from several countries (e.g., samples from Thailand, Taiwan, and Hang Kong) 
whose U.S-bound TOEFL candidates typically score well below 550. Based on 
supplementary analyses (Table 11), both selection-related and English-profici- 
ency related factors need to be considered in an explanatory rationale for 
these findings. 

UGPA is a very important supplemental predictor for U.S. students. 
However, for foreign ESL students potentially useful UGPA/EYA correlations 
were found only in data for students reported to have graduated from a U.S. 
undergradute institution (following Table 11). Although UGPA was reported for 
ESL students by only 22 schools, the results are believed to be general izable, 
due to the strong logical expectation that U3PA/7YA relationships should be 
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lower for students with diverse, intsmational undergraduate origins tha for 
those who completed their undergraduate work in the United States. 



Discussion 

The use and interpretation jf the GMT scores pre sent ed by foreign ESL 
students is complicated by the varied linguistic-cultural jbadcocounds of the 
students. There are differences among students, by country, with respect to 
characteristic backgroun d-pat terns o£ ftxjlish language acquisition and usage 
—that is, differences in Using of initiation, amount, duration, intensity, 
variety, and overall quality of students' English-language ia^veamnt. These 
differences, and related linguistic-cultural variables, aake for iaportant 
differences, by country, in the functional ability of U.S^bound students to 
perform English language verbal tasks of tht type represented by GRB verbal 
items. 

Judging from the findings of this stucV, differences in functional 
ability tend to affect performance _n the MB* program as wall as performance 
on GRE verbal test items: the relative first-year wilMi-school stsnding (mean 
T-scaled FA) of foreign students by country of citizenship was relatively 
closely associated with their relative standing (f-eceled mean) on GMT 
verbal, not with their relative standing on GMT quantitative, which was 
sys erratically more valid as a predictor of PA within various classifications 
of students. 

Performance cn GMT quantitative does not appear to be affected by level 
of English proficiency, very high average levels of ability to perform the 
tasks represented by CMS quantitative items are commonly exhibited by foreign 
.students with limited English-languaye bac k g r o u nd s . This measure appears to 
maintain its construct validity across linguistic-cultural boundaries. 

However, in samples of foreign ESL students, the GMT verbal section 
(like TOOL) appears to be measuring differences in the acquired functional 
ability to perform English language tasks (English proficiency) as such as (in 
addition to) English-language verbal reason ig ability, tho test-construct. 
The amount of test-construct-related vs English-proficiency-related variance 
in the GMT verbal-score distributions c" students from different countries, 
by inference, is largely a function of the extent to which, by virtus of their 
respective heritages (linguistic and cultural) and patterns of English lang- 
uage acquisition and usage, the respective student groups tend to approach 
native levels it fluency in English, m this study, GMT-V/FA correlations 
like those typically reported for samples of first-year MA students by the 
GMC VSS were found only for EEL students (largely from native-English 
speaking countries) and for students with exceptionally high scores on TOEFL 
(over 600). 

Although observed differences in quantitative score means for students by 
country, as v*ll as score differences for individuals without regard to 
country, appear to be reflecting, primarily, valid differences in levels of 
developed quantitative reasoning ability, the relative standing of various 
country contingents in terms of first-year MBA performance (mean T-scaled 
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FA), tended to correspud with their relative standing an the GNAT verbal 
measure 'which by inference indexes differences in English language background 
and proficiency as such as {in addition to] differences in verbal reasa Jig 

ability). 

To the extent that average di ffer ences in FA for students classified irv 
country are affected by differences in "English proficiency,* questions are 
raised regarding the interpretation of obeerved di ffer ences in the average 
first-year grades of studerit* frost different countries. Differences in seen 
fya us.y -efiect quite accurately characteristic differences in the Manifest 
beftavior and acadeedc productivity of students in the* respective mticnal 
groups. However, students who are sorters of nattcorl groups with relatively 
low average facility at ngliih langaege verbal processing ay tend to be 
haqpered in their ability to "show what th^ taw*,* **eed on normal 
evaluations! procedures, wlative to their counterparts with richer relish 
language backgrounds and co rr e sp o n di n gly greater functional Bngiisb-langu<ge 
facility. Given the foregoing in te r p ret ive rationale, questions My be raised 
regarding the "msanim* of average di ff e r ences in fa, aacng grou^^of students 
representing national groups with characteristically different levels of 
English proficienc y such di ff e re nces should be interpreted with .caution* The 
"meaning" of Fftrdiffietinces esong students with cosacn linguistiocultural 
heritages (e.g., fro* the sane country) is not at issue hers. Hie anbiguities 
in meaning alluded to are those associated with the interpretation of average 
differences in FA for national oroups of foreign students, especially b e twee n 
those characterized by atyoically high average quantitative scores and low 
average verbal scores (with lower average Bnglieh proficiency) vs those with 
relatively high levels of Bnglish proficiency (vfco tend to earn better grades, 
notwithstanding lower levels of quantitative ability). 

Foreign ESL students with very hi£i quantitative ability but low English 
proficiency nay acquire more prograat-related knowledge, skills, and 
understandings than they are able to exhibit through their classroom 
participation, performance on examinations, and written work — as typically 
evaluated fay the faculty. It would be useful, on an exploratory basis, to 
employ more intensive, and potentially more sensitive, personal assessment 
procedures to evaluate students' grasp of concepts, understandings, and the 
like. 

These findings suggest that admission practices that favor "otherwise 
qualified" foreign applicants from countries vhose ESL-nationals typically 
exhibit high levels of developed proficiency in English, over those from oJier 
countries, might result in iuproved levels of performance of enrolled foreign 
students on the FA criterion. Mastership in a particular group may provide 
information having pred ctive utilit y beyond that provided fay measures of 
individual performance. For maple, TOEFLEVEL, a variable employed in this 
study as a supplementary predictor of FBI (based entirely on historical 
country-level data— mean TOEFL scores of U.S. -bound nationals, ascribed to 
students from each country) added significantly to prediction of FB I when 
included in 2 battery with GMKF-V, GMKP-Q, and individual scores on TOEFL, 
Questions of policy are beyond the scope of this paper. However, there are 
inportant isou.s of equity involved in the use, in selective admission, of 
predictive background information based solely on group membership. 
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mfllca ti o os for Prediction 

ihe privy aim of this exploratory study *as to assess the potential role 
of selected test and background variables as moderators of the relationship 
between GMT scores and first year perfbcsnoe ifi samples of foreign stud ent s * 
Such an aaeesnsnt «as thought of * constituting a needtd first-step in the 
dsvelw^sent of operational <9MT prediction-systems for foreigp students. 

A major implication of the finding is tfasfc a set of subgroup prediction 
systems for prospective foreign tSL students is Jiksly to be sort effective 
than any general systea. OK/rih correlations toe besp show to be hi0*r, 
as hypothesised, for subgroups of state** c ha r a cteris ed by richer Biglish 
language bac kgr o u nd s and higher levels of toglisb proficiency than for those 
with sore limited toglish backgrounds as indexed -both fcy their national 
origins and their perforaenoe on ^glish-proficdencaf^relatad test variables. 

Classification of students by country of citisaqetaip appeara to have 
premise as the basis for a moderated system* Based on the findings of the 
present study, most of the mock ating effect associated with country of 
citizenship might be realised by a classification setose like that outlined, 
illustratively, below 

Group A: Students from rative Bnglish-speaking countries (prediction 
rules developed for U.S. citizens might be applicable); 

Group B: Students frost non-native English speaking countries those 
u.S.-bound students typically exhibit relatively high levels of toglish 
proficiency: e.g. , students frosi west-European societies *hose linguistic- 
cultural heritages are similar to thoee of U.S. students; students from Asian 
and African countries in vhich English is an official language and/or an 
academic lingua franca, especially in higher education (e.g., Ind ia, 
Singapore, Malaysia, Hong Kong, Philippines); countries for which TOEFUEVL 
typically is 550 plus. 

Group C: Students frost countries without a strong academic English- 
speaking tradition, whose heritages (linguistic, cultural, and educational) 
are moderately similar to those of U.S. students: e.g. , students from South 
and Central America, eastern Europe, Greece, Turkey, Cyprus; countries with 
TOEFLEVL of S25 or higher, that are not elsewhere classified; countries for 
which TOEFLEVL typically is less than 550, but greater then 525. 

Group Dx Students from countries with a very limited English-speaking 
tradition, those linguistic-cultural heritages are not similar to those of 
U.S. students: e.g., students from Taiwan, the Peoples' Republic of China, 
Japan, Thailand, Korea, and Asian countries not In Group B, above; students 
from Arabic-speaking countries; c ountri es not elsewhere classified with 
TOEFLEVL less than 525; for Group D, TOEFLEVL typically is below 525. 

Classification by country (a) introduces direct control for differences 
in relevant cultural, linguistic, and educational background variables, nested 
in countries of citizenship, that are controlled only indirectly by classify- 



er|c 



57 



-43- 

ing students according to scon levels on the test variables and (b) results 
In substantial incidental sorting on the Ehglish proficiency-related test 
variables For example, students m Group 0 were generally quite loir on the 
FVFZ; however, Arabic-epeaking students, also placed in Group D, had Moderate- 
ly hi£i scores on this index. Consideration of the nan test performance of 
student-nationals sight improve ths place— it of particular countries in any 
such classification. 

Alternatively, a classification scheme based solely on the test variables 
might provide the basis for a Mo derated system; classification according to 
score-level on the HVPI, or TOEFL, results in substantial incidental sorting 

on country. 

o FVPI appears to be proadsing as the basis for subgroupiag. It is 
derived from GMNT scores and it is indexed tp the gmecal population of 
U.S. GHKT examinees. Hoc the data wder ctineidecatidn in this study, 
GMflMyFJtfc correlations exceeding r-« .30 (typical for aaaples of U.S. 
students), nere found foe students with hL<fmt aod mediue values 
(two-thirds of all BL itadents); for the remaining studmts aKVFfh 
correlations were relatively low (r - .16, and r - .20 for V/RA and 
Q/FBt, respectively). Moat of the lower-HVPI students had GHKF verbal 
scores that were acre than two standard errors of estimate lower then 
would be expected for U.S. examin e es with onqparaMo OAV quantitative 
scores, and they were disproportionately froa the Alien countries in 
Group O. 

o Classification according to score-levels on TOEFL appears to be 
somewhat less proadsing as the priaary basis for an operational 
moderated-prediction system: TOEFL scores are nbt routinely available 
for all SSL students (alm ost one-third of the JBL studmts in this 
study did not present TOEFL scores), e nd a strong moderating effect was 
evident only for those students with TOEFL scores of approximately 600 
or higher. However, in addition to providing useful information 
regarding the g eneral lavel of Baglish language verbal skills for ESL 
students, TOEFL total score appears to have promise as a supplemental 
predictor of FA for foreign ESL students, especially those wi th l ower 
levels of developed proficiency. Generally speaking, toejvfya 
correlations wpre ccnparable to <3Q&-V/Fih correlations. 



Further Research 

The findings of this study suggest that the formal prediction-rules ^ mul- 
tiple regression equations for predicting FA from GHAT scores and other rele- 
vant test data) for classifications of students such as those suggested above 
are likely to differ— i.e., the regression systems for subgroups such as the 
foregoing are not likely to be oonparable. 

Further research is needed (a) to assess the comparability of subgroup 
regression systems and (b) to investigate the practical utility of a modera- 
ted prediction-system for foreign ESL applicants. A statistical model based 
upon empirical Bayesian concepts, has been applied by Braun and Jones (1961, 
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1962), in studies involving small samples of minority students in several 
schools of a anag ene nt and a number of email graAefte-deperteental samples, 
respectively. Ihis model would seam to be adaptable for application to the 
problam of developing attesting the utlHbjfie aoderateoH*ediction systea 
for foreig n Ksi a jalT cenbtt fib m the eaen y setionja^rogae^ saeples of 
f o ra iga students w vexaaw H^geioemaev ' llBav-'deWsloBBawte' of an operational 
predictioB eystea with prooram-level applicability v&U. call for the ose of a 
statistical aodal that ie oansblerof treating data foe seall saaplso froa a 
relatively large number of programs. 

xn d es igni n g further reeserch on aoderated-predicticn systems for foreign 
esl students, it is iaportant to take; into aeons* and distinguish between 
subgroup differences in OK/Dtk correlations that are due to the moderating 
effects of toglisbgr o^^ agcj^or^ Q^^ b^ij^MB^mtiMi^^ and d iff erence s 

example, the level of co rrelati o n s tended to br lew for^udents im'^arger^ 
sore selected esaplea, than £fot those ii*vaeiller, 4eee highly selected 
sasples, regardlsss of the status- of students on particular andetator vari- 
ables. Selection effects on p re di c to r/ nex ralatkratdps say be pronounced 
for soat foreign naticnala (for example, thos*with very low HYPI values, 
characterised by vary high quantitative ecorea arid relatively low verbal 
scores) than for others (see labia U, and related dis cu s sion ). Xn tubus 
research it would be useful to have inforsation regarding instituticnel 
practices in evaluating the acadaeic qualifications of foraiga applicants. 

It is iaportant to recognise that the schools that participated in this 
study are not necessarily representative of all schools offering MB* prograas. 
The sasples of foreign students had scores on Offff that-ware well above the 
average for the general OMP-oandidnbs population; students with TCEfL scores 
were very highly selected relative to the general GHMMOBL and TOEFL 
populations, respectively, ihus, it ssaas reasonable to infer that the level 
of English proficiency characteristic of this laapl u probably aaa higher than 
that for all enrolled foreign ESL students. Xn the design of future research 
involving classifications of students according to relative English-profici- 
ency levels as indexed by TOEFL MO HKFI, it is iaportant to take into account 
the general level Of p roficiency in the laaplsi involved— the ' icular 
cutting points on TOEFL and RVPI used in this relatively highly 'icted 
sample sight not be strictly applicable in sanplee with lower average ^vels 
of English proficiency. 

The results of the present study, like those of studies of the 
characteristics and the test ptrforaence of foreign nationals taking the GRE 
General lest (Wilson 1984a, 1984b, 1982c), and of previous studies of the 
impact of language background on CHMF p erfor mance (Bowers 1980; Wilson 1982c), 
strongly suggest that English language "verbal ability tests" are not 
measuring the saae construct in samples of native and non-native English 
speakers. Verbal score differanosa bsttaaen U.S. oxaalnese and randomly 
selected foreign ESL e xa m in ee cannot-, be aasussd bo reflect valid differences 
in developed verbal raaaoning ability. TMe ia a consideration that should be 
weighed carefully in designing and interpreting the results of validation 
research that eight involve pooling verbal test data for U.S. and foreign ESL 
students (or examin e es ). 
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A-l. Distribution of the Stud/ Sanple ty Country of Citizenship 



A-2. intarcor relations of means on stud/ Variables for s tuden t 
Contingents f roa 32 Countries 
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Distxtbutlon of the Study Sample by Country of 
Citizenship 
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Note: Throughout this study, Independent nation-states, dependent terri- 
tories and other geopolitical entities are all referred to for con- 
venience as countries of citizenship . 
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GNAT-V 
GMAT-Q 
GMAT-T 
VSSCCNP 
SI < 

8IRTHVR 

USUG*L 

RVPINO 

TCFI-MN 

TOFT! TCT 

VFSTGf 



GMAT-V 

I. 0000 
0.1977 
0.8246 
0.6141 

-0.2039 
0.0725 

•0.206 f 
0.831? 
J.5949 

II. 73*1 
•0.0192 



GHAT-0 

0.1977 
1.0000 
0.7124 
0.8948 
0.0315 
-0.0367 
-0.4527 
•0.3806 
»0.tt56 8 
-0.0074 
U.2861 



GNAT-T VSSCONP 



SEX 8IRTHVP 



0.8246 
0.7124 
1.0000 
9.9488 
-0.1183 
-0.0143 
-0.3957 
0.3737 
0.4039 
0.5318 
Or 14*4 



0.6141 
0.8948 
0.9488 
1.0000 
-0.0642 
-0.020? 
-0.4542 
0.0715 
0.225U 
0.3304 
0.2172 



-0.2038 
0.0*15 
-0.1183 
-0.0642 
1.0000 
0.03?2 
0.2128 
-0.2124 
-0.2621 
-0.2854 
-0.2843 



0.0225 
-0.0367 
-0.0143 
-0.0203 
0.03*2 
1.0000 
0.0432 
0.0428 
0.0384 
-0.0644 
0.0948 



USUGM 

-0.2067 
-0.4577 
-0.3957 
-0.4542 
0.2128 
0.0432 
1.0000 
0.0588 
-0.3451 
-0.1825 
-0.7810 



RVPINO 

0.8312 
-0.3806 
0.3731? 
0.071% 
-0.2126 
0.04? 8 
0.0588 
1.0000 
0.5933 
0.7001 
-0.1773 



TOEFMN TOEFL TOT 



0.5949 
-0.0568 
0.4039 
0,2250 
-0.2621 
0.0384 
-0.3451 
0.5933 
1.0000 
0.7127 
0.1970 



0.73P1 
-0.0074 
0.5318 
0.3304 
-0.2854 
-0.0644 
-0.1925 
0.T001 
0.7127 
1.0000 
0.0261 



*ES?OF 

-0.0192 
0.2861 
0.1404 
0.2172 

-0.2843 
0.0948 

-0.7810 

-0.1773 
0.1970 
0.0261 
1.0000 
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B-l. Preliminary Report to Participating Schools 

B-2. Plot of OW Verbal and GMNT Quantitative Means 

for Smaller, Mediuo, and Larger Samples, indicating 
Relationship between saeple Size and Degree of Selection 
on the GMKF 
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THE VALIDITY OP GMAT SCORES FOR PREDICTING FIRST YEAR 
AVERAGE FOR FOREIGN STUDENTS IN MBA PROGRAMS 

Educational testing Strrlca 
Princeton, NJ 08541 



To: Study Coordinator. Fro.: fcnnathM.Wll.on 

Subject: An Interim Report Data: March i, 1984 



Due to dlfferancaa between foreign nationals and O.S. d tisane in 
lingul.tlc, cultural, end educational background* » Information regerdlng the 
predictive validity of tredltlonal academic predictor., auch aa GMAT acore. 
end the undergraduate CPA baaed on .ample, composed primarily of atudenta who 
are U.S. citizen, ahould not be assumed to be applicable for foreign nationals 
who apply for admission, especially those for whom English is a second 
language. To enhance understanding of how GMAT .cores and' other Information 
about foreign atudenta relate to their performance during the first year in 
MBA programs, all schools of management ware Invited, in March 1983, to 
participate in an essentially exploratory cooperative study by supplying a 
standard aet of data for foreign national, who enrolled for the flrat time, aa 
full-time student., In fall 1982 (and fall 1981 If needed to augment aample 
size). 

The data requested were as follows: 

o OiAT Verbal, Quantitative and Total scaled scores 

o Undergraduate GPA (optional) 

o Total score on the Test of English as a Foreign Language (TOEFL). 

if available 6 

o Sex 

o Blrthyear (Inversely related to age) 

o Undergraduate origin (U.S. va other) 

o Native language 

o Country of citizenship 

o First year everage 



A total of 59 schools supplied data (for a total of about 1900 foreign 
students), most of them for the 1982 entering cohort of foreign students 
only.* All 59 schools supplied GMAT scores, the first-year average (FYA), 
birthyear, native language, end country of citizenship. Attesting implicitly 
to the problem of evaluating the undergraduate academic performance of foreign 



* One additional .chool supplied data for cohorts entering at times and in 
years other than those specified for the study. Data for this school are not 
Included in this summary report. 
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applicantt only 21 schools supplied data on tha undargraduaca GPA 
or UGPA. TWo schools did not provide data on tha sax of students and four did 
not indicate the undergraouate origins of studants. Finally, two of the 59 
schools did not report scores on TOEFL Ibtsl score for any student-- among the 
57 schools that did report TOEFL scores, the percentage of students for whom a 
percent*" r,p °" ,d ran ** d downw * rd from about 98 percent to approximately 17 

About this Be port 

This interim report present j selected results of stsndsrd statistical 
'« ? M !« 0f data for for « lgn ESL ggljlsh second i.n f .. g .\ .tudente fro a each 
°V5? 59 P«"clP«ting schools, namely, <*> Infomstlo! regsrding thl ul.1 
and distribution of score* on OUT and other variables, and (b) coefficients 
of correlation Indicating the interrelationships among GHAT scores, TOEFL 
Totel scores, first-year average (Fla), end UGPA (if provided).* 

Twenty-five of the 59 schools partlclpstlng in this study had participated 
prior to June 1983 in the GMAC fclldity Study Service (VSS) et ETS by 
submitting GMAT scores, first-year performance (FXa), and other data for all 
firet-yeer MBA studies. For these 25 schools, findings froa their previously 
studied genersl VSS samples provide e basis for comparison with findings for 
foreign -ESL students in the present study. 

Emphasis in this Interim report is on trends In selected findings across 
ell schools rsther than on tha specific findings for your school whicifare — 
attached to your copy of this report. The resson for this has to do with 
sample size. As may be seen in Figure 1, the ssmples of foreign ESL students 
by school ere sll quite small by usual validity study standards. The median N 
for the 59 samples is 26, with s range of Ms between 6 and 77. Only three of 
85 general first-year samples studied by the GMAC VSS during the academic 
years 1977-78 through 1979-80 Included fewer than 77 students and the mean 
sample size was 175 (Hecht 4 Fowerc, 1982). Findings for single small samples 
do not provide rellsble bases for generalization. However, given comparable 
data (GMAT scores end FYA) for s relatively large number of small samples it 
is possible to drew some useful inference, regarding tha relationship of GMAT 
scores to first-yesr performance by examining trends in the level of GMAT/FYA 
relationships over sll ssmples. 



*The correlation coefficient is s generslly familiar index of association cr 
covariation between variables. The size of e coefficient indicates the degree 
or closeness of aasoclatlon between two variables on e scsle renging from .00 
(indicating no relationship at all), through 1.00 (indicating either a 
perfect positive or a perfect negative relationship. If the relationship is 
positive, higher standing on one variable tends to be associated with higher 
standing on the other; if negative, higher standing on one variable is 
associated with lower standing on the other and vice verse. 
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7 

6 24558 
5 03 
4 034499 
3 02236679 

2 0O01112222334555677778P? 
1 1 11234456 
0 I 66799 

No. school* (59) 
Mdn N 26 

Note. Combine number in the first column with 
•ubsequent entries to read sample sizes. For 
example, the largest Simple Included 77 students, 
there were three samples with 20 students, etc. 

Figure I. Distribution of samples of foreign-ESL students 
by size 
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Accordingly, it la important that you via* the attached findings for your 
school primarily aa descriptive of ralationahlpa laong variables in ono 
ralativaly snail aaapla. In avaluatlng tha findinga for your school and 
trands in corrolatlona between taat scorss and tXA acroaa schools, it is 
useful to kaap in aind the following ganaral points about predictive validity 
data. 

1) Evidanca froa validity studlea that nave baan conducted extensively in 
undergraduate and other set tings, involving measure* of developed abllltlea 
(e.g. , verbal and quantitative reasoning) and aeaeuree of acadeaic perform.net 
(••*•• t grade point averages), aa wall aa aore general evidence of the 
positively lnterreleted orgenisetlon of huaan abilities," leads to the e priori 
expectetlon that validity coefficients for acadeaic predictors (such as 
ataodard adaiaalona teats or VQfk) and acadeaic criteria (such as FYA) should 
tend to be poeltlve. Za eeeenca, it la reasonable ta assuee that individuals 
with better qualification " (at reflected la their parte acadeaic record end 
their scores on aeasures of developed verbal aad quantitative abilities) 
ehouJd tend to be somewhat "better students" (as reflected in faculty 
essessaents of their perforaence). Hegetlve coef ficiencs for academic 
predictors end criterie are, therefore, properly perceived as theoreticelly 
anomalous. When observed, they, indicate the need for further exploration and 
analysis designed to lllualnete the particular drcuaatances involved. In 
email samples such as those under consideration here, negative coefficients 
ere most probably due to slapls sampling fluctuation. 

2) Generally apeeklng, the else of velidity cocfflclcnta for variables 
used in selection tends to very Inversely with the degree of restriction of 
range of telent in seaplss being studied. In eoeples in which students ere 
homogeneously high on en edalsslons aeaeure, the reletlonahlp between scores 
on that measure and meesures of perforaence in the progrea tenda to be lower 
than would be obtelned if the achool admitted atudenta representing the full 
range of talent (e.g., e group repressntstlvs of ell college ssnlors aspiring 
to MBA programs). 

3) The foregoing points ere relevant to any evaluation of reporttd 
validity study results. In eveluatinr the school dots for samples of foreign 
students from non-native English specking societies it Is important es well to 
«-nn«<rf. r the potsntlsl sttsnustlng sffect on the reletlonshlp between stsndard 
predictors end critsris (e.g., scorss snd FTa) of differences in the 
linguistic, culturel, end educational baikgrouo-'i of the atudenta in the 
particular samples being analysed. r»*se effects, which will be examined in 
the pooled dete analysis, csnnot be evelue:ed directly in the data reported 
herein. 

Selected Fltdl^gs 

Figure 2 shows distributions of the 59 school means on GHAT verbal and 
GMAT quantitative, respectively, for foreign-ESL students. Verbal means 
ranged from 12 to 32 (median cf 24) while quantotltlve aeons ranged from 16 to 
J , (med ™ 0 !, 35) ' For P« r *P«ctive, the i^ans for all U.S. examinees tested 
during 1980-81 on these two measures v#re 2rt.3 and 26.8 for verbal and 
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GMAT Verbal 



GMAT Ouantitative 



4 




4 


0000111333 


3* 




3* 


5555556666667888899999 


3 


00012 


3 


000001112223334444 


2* 


555555556777788899999 


2* 


566699 


2 


00C0001 I 1 1 1 222222344444 


2 


44 


1* 


558888889 


1* 


6 


1 


2 


1 





No. means (59) (59) 
Median 24 35 



Note. Combine number in first column with subsequent entries to read 
sample means rounded to whole numbers. For example, there were three 
verbal means of 30, one of 31, one of 32, etc. 



Figure 2. Distribution of sample means on GMAT Verbal and Ouantitative 
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quantitative respectively (GMAC, 1982). It is evident that these foreign- 
student saaples have been very highly selected on quantitative ability—only 
seven (11.9 percent) of the samples had average scores on GMAT quantitative 
that were lover than 29. By mj ef contrast 9 42 (71*2 percent) had verbal 
■sans lover than 27 on GMAT verbal. lovever, the verbal aedian of 24 is 
higher (by alaoat one-half of a standard deviation) than the Man of 
approxiaatly 20 obtained by aJ^ foreign nationals, without regard to language 
background* vho took GMAT during i»77*lf79, vhose quantitative wan (27) was 
equal to that for al\ GMAT exaalnees tasted during that period (Wilson, 1982b; 
Fovers, 1980). Thus, it asy be concluded that relative fo #11 foreign 
nationala the student in these saaples ware highly selected on both verbal 
and quantitative , although aore highly selected on quantitative than on verbal 
ability. It is iaportant to note, la passing, that the aaaa quantitative 
perforaance of foreign nationals taking GMAT, is coaparable to that of U.S. 
citizens while the verbal aaan tends to be considerably lover* 

Figure 3 shovs distributions of siaple correlation coefficients between 
designated predictora and FIA for the 59 schools. Ihe predictors are GMAT 
scores (verbal, quantitative, and total), and a standard coaposite of GMAT 
verbsl and quantitative scores (Q ♦ *6V). Separate distributions of 
correlations are shovn for the 21 schools that reported DCPA. Ihe veighting 
of the verbal and quantitative scores in the standard coaposite (namely, Q + 
• 6V) reflects the ratio of average vaights for these tvo scores derived in 
analyses of data for all entering students froa the 25 study schools that 
previously participated in the GMAC Validity Study Service (VSS) at ETS. 
Several features of these distributions are noteworthy, 

o Despite the fact that the saaples under consideration are heterogeneous 
with respect to linguistic, cultural and educational background 
variables, the observed correlations for the test scores (and the score 
composite) with FYA are preponderantly positive* 

o Based on the median valuee, for "all schools" the GMAT quantitative 
tends to be a better predictor of FYA (aedian • .30) than is either 
GMAT total (aedian • .27) or GMAT verbal (aedian - .16). Ihe finding 
that the aedian coefficient for GMAT total is less than that for GMAT 
quantitative alone aay be understood most simply in teras of (a) the 
lover median validity for GMAT verbal than GMAT quantitative and (b) 
the fact that the GMAT contains aore verbal iteaa than quantitative by 
a ratio of approximately 3 to 2. Thus, the less valid predictor 
(verbal) ia weighted aore heavily than the aore valid predictor 
(quantitative) in the GMAT total score* 



o For the 21 schools providing UGPA, the aedian coefficient was .12; the 
distribution of coefficients for (MAT scores for these schools was 
about the same aa for all schools. Thus, UGPA tends to be somewhat 
less closely related to FYA than GMAT scores in these samples of 
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All schools 



Schools provfdlng OCFA 



GHAT-? 



CMAT-Q 



(MAT Total 



Q ♦ .6?* 
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No. coeff. (59) 
Mdn cocf*. .16 



(59) 
.30 



(59) 
.27 



(59) Ho. coeff . (21) (21) (21) (21) 
.30 Hen coeff. .16 .30 .31 .12 



Note. In these distribution., the Initial digit of a coefficient la indie* to* In the ftret colon* end the 

second digit of each coefficient la recorded in the second and nveeeoneet colon**. TWs f for enseals, 
there was one CHAT-V coefficient of .75 f one of .65* one GHAT-Q coefficient of .78, one of .53 end 
one of .69; etc. 

*Thle is s composite of GNAT verbal and ensntitstlve scores, freighted according tw the ratio of tynlccl 
weights for theee scores ss derived in total sonnies of students from 23 study schools that previously 
participated in the GHAC Validity Study Service. 



. lgure 3. Distributions of correlations of designated predictors with FYA 
by school 
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forelgn-ESL studsnts all of which sre aads up of Individual* with 
dlvtrat undargraduata educational origins—only about 23 ptrcent of all 
foralgn ESL studsnts wra graduetes of a U.S. undargraduata school.* 

Uasful perspective for evaluating tha distributions of corralatlons betwen 
OUT acoras and FXA for foralgn ESL studants Is provided In Plgura 4 
which provldas (a) distributions of OUT vaUdity coefficients in ssnplcs 
conposcd of all studants (0.8. dtlxans only and/or U.S. cltlsans and 
non-cl titans) basad on 83 studios completed by tha (MAC 988 during tha 
thraa-yaar parlod 1978-79 through 1980-81 and (b) similar distributions for 
all-studant snd foreign-* SI studants, respectively, for tha 25 study schools 
for whoa "ell studant" validity studios vara completed by tha (MAC VSS during 

the five-ye ar parlod 1978-79 through 1982-83 and (c) data on saapla 

charactcrlstcs. Several features of thosa distributions warrant cownent. 

o The sversge ssaple size for the 85 regular VSS ssaples was shout 175, 
snd that for reguler ssaples Irosi tha 25 study-schools was 181, ss 
coapcred to 32 for foreign ESL students only. 

o The ell-student ssaples fron the 25 study-schools that previously 

pertlclpented in the (MAC VSS tended to be aore highly sslected on both 
2* OUT verbel end (MAT qusntltstlve than those from the 85 VSS 

b pertldpants generelly (compere renge of school nesns on verbel end 

^ quentltetive for the generel VSS psrtlclpstlng group snd ths joint VSS 

< - snd study participants), 

jo 

-< The aedlan correlations between FYA end verbel snd quentltetive scores 

O ln th • W VSS " a11 »t«dent- ssaples were .25 end .30. respectively. However, 
O to l th « "ubgroup of 25 schools the coapareble "all student" coefficients were 
-H * 18 and ,28 (•oaswhat lower th^n typical coefficients for ell 85 VSS schools) 
end for this ssne subgroup of schools ths coefficients for forelgn-ESL samples 
> ^ only were .20 for verbsl snd .25 for qusntltstlve (es coapsrsd to .16 snd .30 
for sll 59 forelgn-ESL ssaples). 

On balance the findinge summarized la Figure 3 end figure 4 suggest that 
for foreign ESL ssaples thst srs hstsrogeneous with respsct to national origin 
(a) the correletlon between (MAT quentltetive scorss snd flrst-yeer CPA tends 
to be conperabls to thst observed for ell flrst-yeer studsnts (predoalnantly 
samples of U.S. citizens), (b) tha correletlon of (MAT verbel scorss snd UCPA 
with FTA tends to be somewhet lower then thst observed for U.S. citizens only, 
s result that might be expected given the heterogeneity of the forei< -ESL 
population with respsct to linguistic, cultural, snd sducstlonsl bac. -ound 
variables that might be expected to ettenuete the reletlonshlp with FiA of 
verbal test scores and Indies* of past ecedeoic performance. 



CO 



*In several instance UCPA was missing for s substantial number of students. 
Reasons for this srs not known. Accordingly limitstions of the UGPA date 
should be recognized. 
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Oiotributioa of VolMlty Coefficient* 
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(1978-79 - 1960-61) 
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Participants in both VS8 
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Figure 4. Cowparative distributions of correlations of CMAT scores for FYA for 
"all student 11 and fnrelgn-KSL students only. 
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It it worth noting here that the principal difference between tht 
distribution* of CMAT vslidity coefficient! for the 25 joint 7SS-and -study 
participants appears to ba cha graatar ranga of coafficiants for tha auch 
sasllsr forelgn-ESL saaplas than for tha larger "all student " staples. As 
indicated earlier, such a difference sight ba explained in ten* of 
differences in the degree of snapllng fluctuation around similar population 
correletlon values for "all student" and foreign student saaplas. Major 
attention should be focussed an trends in the level of coefficient*. 



In Figure 4, as previously noted, the ranga of observed correlations is 
conslderebly greeter for the snail forelgn-ESL saaplas than for tha Urge 
ell-student seaples. However only a selected subset of foreign ESL seaples is 
represented. In order to assess verietlon in Observed correletlons of CMAT 
scorss with FIA for forelgn-ESL students in relation to saaple size, the 
distributions in Figure 5 were tabulated. 

This figure shows distributions of quantitative and verbel score correle- 
tlons with FIA for 13 seaples with V less than 20, 24 saaples with > ■ 20-29 
end 22 eeaples with M of 30 or greeter. It is noteworthy that for tha 
quantitative test, aedlan validity tenda to vary Inversely with seapls size 
cetegorlee, being highest for the sasllest seaples (r • .39 for > < 20) and 
lowest for the 22 largest saaples (r - .23, tha saae as for the joint VSS end 
study subgroup), with the uedlen for saaples in the alddle slze-renge felling 
in between (r - ..1). Trends for verbal score validity, on the other hand, 
ere not systcaatlc: the verbal score aedlan for the saslless size-category 
(r • .25) is higher then that for the lergest slze-cetegory (r - .19), but the 
aedlen for the middle-size category is only .07. 

In evaluating this out coot, it is laportent (e) to know that the lerger 
aampl«s ware wore highly stlected on CMAT quentltetlve ability then the 
smaller ones end (b) to recell the generel principle thet velldlty 
coefficients for e predictor tend to decreese as the degree of prior selection 
on thet predictor lncreeses. 



Totel scores on the Test of English as a Foreign Language (TOEFL) ware 
reported for one or wort students by 57 schools. The aedlan percentage with 
TOEI'L Total was about 68; however, the ptrcentege of students with TOEFL 
scores varied considerably (froa 17 percent to 98 percent). Accordingly, the 
number of atudents with TOEFL and FTA was systeaatlcelly less then the r umber 
with CMAT scores end FTA. 

The median of TOEFL means for the 37 schools was approximately 380, and 
the mean for all TOEFL-takers without regerd to school was 388. Sample means 
ranged from 513 to 617, but the greet majority of samples (about 85 percent) 
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hac' TOEFL meana of approximately 350 or higher and almost ona-thlrd of the 
sample aeane vara 600*. ror perspective it la uaeful to not a that tha TOEFL 
mean for aU OUT/TOEFL examinees daring 1977-71 (Wileon, 1982) vat 353, and 
' or a11 J'S.-graduate-acb^l-btmnd TOEFL-eakara during 1977-79 (Wilson, 
1982a) was only 511. Thus, tha foreign-ESL TOBFL-takere ii tha eaeples under 
consideration la thla study ara quite highly selected In taraa of aeaeured 
Englieh proficiency aa indexed by TOEFL Total, The aedlan correlation between 
TOEFL Total and FYA for 56 sample, aaa .22. Flgura 6 .how, two aata « 
distribution of the observed nefficiente, one involving four elae-cetegorlea 
and the other only tan, to provide additional empirical perspective on 
variability in aaapUng fluctuation of coefficients due to aaaple else. 

In evaluating theae correlation, it la Import ant to keep in aind that 
they repreaent relatlonahlpe in selecte d sub s sspln of foreig«SL etudente 

»choole and hence ehould not be eoapared directly with the 
distribution of coefficient, for OUT verbal or qnntitative which arf baaed 
on all ESL atudenta In the reapectlvc school saaplee. 

The Findings for Tour School 

Descriptive statistics for the aaaple of foreign ESL atudenta froa your 
school are provided below on the following varlablea: 

FYA (first-year average) 
CMAT verbal 
GHAT quantitative 
CMAT total 

VSS Coapoaite (Q + .6V) 
TOEFL Total (If available) 
OGPA (If available) 
Optional variable (If supplied). 

The nuaber of students with observatlona on each variable la shown, .long 
with means, standard deviation , and mlnlaua and aaxlaua valuea for each 
vsriable. A table of intercorrelations of all the varlablea Is also shown. 

The number of atudenta, aa indicated in the output below, may be less than 
the number of atudenta Included on your basic data roster. Thia will be the 
case if (a) any native-Engliah speaking students were Included In your 
sample— for this preliminary analysis, these students were not included, (b) 
there were missing observatlona on eaaentlal varlablea for any student (e.g., 
CMAT scores, FYA) or (c) there were values on the roster for any variable that 
were inconsistent (e.g., beyond the range of valuea specified for a variable). 

Please review the general Interpretive considerations outlined on page 4 
of Che report. Remember that the findi-w reported below are based on a very 
small sample by usual validity study stt.Jards. It would also be useful to 

Eem^I^Si**?* ££!S t,d ln J Fl « urM 5 . that show how the relationship 
between a predictor (CMAT quantitative) and a criterion (FYA) tends to be 
lower for more highly selected samples and higher for samples that are less 
highly selected on the predictor. It is reasonable to assume that if it were 
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Figure 5- M-tri!wtl0M 0 f correlations of CMAT scores with FYA 
by size of sample: Foreign-ESL students, 59 schools 
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Figura 6, Distribution! *f correlations of TOEFL Total 
with FYA by aample size 
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Nott. Thrtt schools did not rtport TOEFL scorss for any student. Vldsly 
varying proportions of students with TOEFL scorss srs rcprtssntsd In tht 
rsspsctlvs saaplss. Hs with TOEFL scorss rsnged from thrss to 62. 

cSet 1 distributions show trends across four saaols-slzs cstsgorlcs, vhlls 
* Sst 2 shows distributions for ths two snallsr end tht two lergsr sampls-siie 
cttsgcriss as wsll as the distribution for all schools. 

#Thsss two coefficients era sach based on H - 3. 
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sonehow possible to obtslo for your school a very largo similarly salactad 
sanpla of foreign ESL students, the Magnitude of validity coefficients for 
GMAT verbal and quantitative, respectively, in that seaplc would eost likely 
be soaevherc between the aedlcne reported for other slallarly selected seaples 
end the values reported below for your school— and given e very lerge eeaplc 
it is unlikely that the validity coefficients would be negative. 

You nay find that the correletlon between GMAT verbal and quentltetive is 

?! 8 rJl V ! 10 1 TLVF lM i th i* — th * ^ la 22 •* «»• 59 seaples studied). 
In the general GMAT population, the correlation between V and Q is in th* .55 
- .57 range; for ell fcreign-ESL GHAT exaalneee a correlation of .30 nay be 
representative of the reletlonahlp WLleoo. 1982b). Ihe correlation between 
these two predictors tends to be lower in highly selected seaples (for 1, 767 
ESI students in the present study it is only .295 as coapered to .50 for ell 
foreign ESL GMAT exeainces). Given the saell size of the saaple, the observed 
negetlve coefficient any be due to slaple seapllng fluctuation. However, 
negetlve reletlonshlps between these predictors any reflect, In pert, the 
effects of coapensetory selectlon-i.e., requiring very high performance on 
one predictor if perforaance on another or others is very low end a tendency 
to screen out candidates who ere very low on both or all predictors. 

Your essistance in this cooperetlve endeevor is eppreclated. Do not 
hesitate to call or write if you heve questions about this report or the data 
for your school. 

Ksnneth M. Villon 
(609) 734-5391 
Educational Testing Service 
Princeton, NJ 08541 
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C-l. Raw Score and T-Scaled Means on Study Variables 
for the analysis groups in Table 1 

C-2. Scatterplot it CHAT Verbal end TOEFL Scores for 
Foreign-ESL Students 

C-3. Scatterplot of CHAT Verbal and CMAT Quantitative 
Scores for Foreign-ESL Students 

C-4. Scatterplot of CMAT Verbal and CMAT Quantitative 
Scores for Foreign-ESL Students 
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41.00 
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26.46 


64 


56.40 


7.72 
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44 
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150.00 
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53.40 
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44 
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9.45 
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52 
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64 
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0.77 
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0.27 


45.00 


41.00 


227 


57.0? 


2.42 


0.0 


1.00 


211 


0.10 


0.34 



NIN 

7.00 
14.0* 
140.00 
27.40 
<4.f4 
449.00 
405.00 
0.0 
1.00 
45.00 
0.0 



40 

57 



477 
I 
2 
414 
I. 



OtHCIIS>»950l 2214 



OfNMS<550l2Jl 4 



TOJAt 0SU24I 0 



42 
42 
42 
42 
42 
42 
14 
42 
42 
42 
40 



NC AN 

24 .30 
10.45 
449.12 
45,00 
41.12 
59,3.00 
601.75 
O.JO 
1.19 
56.05 
0.30 
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10.51 
20.51 
45.25 
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0.46 
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11.00 


50.00 
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74 


465.03 


101.11 


210.00 


730.00 


72.00 


74 


45.97 


12.99 


16.40 


74.20 


67.00 


74 


17.27 


11.61 


10.49 


70.07 


630.00 


74 


514.12 


20.54 


452.00 


544.00 


670.00 


36 


549.50 


30.41 


491.00 
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1.00 


74 


0.49 


0.50 


0.0 


1.00 


2.00 


74 


1.27 


0.44 


1.00 
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74 
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40.00 
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9.il 


25.il 


69.36 


• 3 


47.71 
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9.10 
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7.90 
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